Click Here to Install Silverlight*
United StatesChange|All Microsoft Sites
MSDN
|Developer Centers|Library|Downloads|Code Center|Subscriptions|MSDN Worldwide
Search for


Advanced Search
MSDN Home > MSJ > September 1997
September 1997

Microsoft Systems Journal Homepage

Under the Hood

Code for this article: Hood0997.exe (48KB)
Matt Pietrek is the author of Windows 95 System Programming Secrets (IDG Books, 1995). He works at NuMega Technologies Inc., and can be reached at mpietrek@tiac.com or at http://www.wheaty.net.

It's not every day that I end up working with assembly language, API interception, the Internet, and Visual Basic® all in the same project. While API spying is up my alley, what the heck am I doing writing Visual Basic and Internet code?
One night, while sitting in the spa, it occurred to me that I wanted to more actively monitor the mutual funds in my retirement plans. There are a lot of great Web sites out there that offer free stock and mutual fund analysis and charting. My favorite is the Microsoft
® Investor site (http://investor.msn.com), but since my personal T1 line hasn't arrived yet, it's painful waiting for my 28.8Kbps modem to grind through downloads.
Also, being a programmer, I naturally wanted the raw data so I could do my own custom analysis and charting. While I can easily use my browser to find the prices of each fund I own, it would be a real pain to manually transcribe all the prices from the browser page. Even worse, the process would need to be repeated every day. No, I wanted a program that used the Web to retrieve stock or mutual fund prices without requiring me to do anything manually. No mouse clicks, no typing. Just run the program and my personal database of fund prices is updated.
Drawing on the vast emptiness of my Internet programming knowledge, it occurred to me that, when using a browser to get an online quote, it's really just an HTTP transaction between your browser and a server somewhere. By determining what data a browser sends and receives when you submit a quote request, it's possible to create a program that mimics these interactions, thereby receiving the same data that a browser would. Luckily, my last remaining bit of Internet knowledge was that Microsoft Internet Explorer (IE) 3.0x uses WININET.DLL—a Win32
® system DLL that provides a high-level layer over the HTTP, FTP, and Gopher protocols, sparing you from the nastiness of Windows® socket programming.
By observing the calls made to WININET.DLL while retrieving a quote, it's possible to create a program that makes similar calls to the WININET APIs, but without the overhead of an entire Internet browser. Of course, the ability to watch what IE is doing is useful for many things beyond mere stock quotes. For example, I found it extremely interesting to watch how IE did things like downloading Java classes, using cookies, and caching the most recently downloaded pages and image files.
Longtime readers of MSJ may recall an article I wrote a while back that presented a generic API spying program called APISPY32. While theoretically I could have used APISPY32 to monitor calls made to WININET.DLL, there would have been a variety of technical hurdles that I won't go into here. Instead, I opted to use a more classical method of API spying. I called the resulting code WininetSpy.
The core of WininetSpy is a DLL that shares a common name with WININET.DLL and exports many of same functions as the Microsoft-supplied WININET.DLL. Each exported function in my DLL performs whatever logging is needed in addition to calling its corresponding API in the Microsoft WININET.DLL. Theoretically, my version should export a stub API for every API in the Microsoft WININET.DLL. But not all of the functions in WININET.DLL are documented, so it would be difficult (but not impossible) to create stubs for these undocumented APIs. In addition, I couldn't find any programs that used the Unicode version of the WININET APIs. Therefore, I was lazy and only provided logging stubs for the WININET APIs that are actually used by IE 3.0x.
Two system-supplied DLLs are layered between IE and WININET.DLL: MSHTML.DLL and URLMON.DLL. By combining the list of WININET functions imported by these two DLLs, I came up with the list of functions that my WININET.DLL had to export. Important note: the WininetSpy code was tested extensively on IE 3.02, using Windows NT
® 4.0. If future versions of IE import additional WININET functions, IE will likely stop functioning if my WININET.DLL is being used. The fix would be to add stub logging functions for the appropriate new WININET APIs. Alternatively, you could just remove my WININET.DLL from its installed location, and everything should work correctly afterwards. You'll see why momentarily.
At this point, you probably have two concerns about how my WININET.DLL fits into the picture. First, aren't there restrictions about having two DLLs with the same name in a process? Second, how can I force the system to connect MSHTML.DLL and URLMON.DLL to my replacement WININET.DLL rather than the system-supplied WININET.DLL? The answer to both comes down to location, location, and location.
Unlike 16-bit Windows, Win32 doesn't care if you have multiple DLLs with the same name in a process address space. When keeping track of loaded DLLs, Win32 operating systems use the DLL's complete path as its name. This is different than 16-bit Windows, which uses the base file name of the DLL (such as WININET) as the module name. So you can have two copies of WININET.DLL loaded as long as they're in different directories. The tricky part is getting them both loaded and everything hooked up properly.
By examining the documentation for the Win32 loader (essentially, the LoadModule API), you'll see that the loader first searches for DLLs in the directory where the application's executable resides. This is perfect for what WininetSpy needs to do. By dropping my replacement WININET.DLL into the same directory as IE (IEXPLORE.EXE), I can force my WININET.DLL to be loaded rather than the Microsoft WININET.DLL (which is in the system directory). Once my WININET.DLL is loaded, it's a simple matter to call LoadLibrary to load the real WININET.DLL. Let's look at some code now to see what I've just described.
Figure 1 shows the code for WININETSPY.CPP, which compiles into WININET.DLL using the supplied makefile. Most of the code is boilerplate API stub functions that perform the logging before calling the real WININET APIs in the system's WININET.DLL. I'll describe them later. For now, concentrate on the DllMain function near the top. The code executed when the DLL loads (inside the DLL_ PROCESS_ATTACH if clause) first disables thread notifications by calling the DisableThreadLibraryCalls. My WININET.DLL doesn't need to know about thread creation and termination, so this call tells the system not to bother calling my DllMain for thread-related activity. Next, DllMain attempts to load the Microsoft-supplied WININET.DLL by calling LoadLibrary with a complete path for the DLL. My code assumes that the system WININET.DLL will be in the Win32 system directory, which it locates via the GetSystemDirectory API.
If everything goes as planned, after the LoadLibrary call returns the system WININET.DLL is loaded and ready to go. The next major task is to look up the address of each of the APIs exported by the system WININET.DLL. As you might expect, I do this by calling GetProcAddress on each exported API. Since WININET.DLL exports well over 100 functions, you might expect to see a whole bunch of GetProcAddress calls somewhere in WININETSPY.CPP. You won't find them, though. The closest thing you'll find is this:

 #define SPYMACRO( x )  \
     g_pfn##x = GetProcAddress( hModWininet, #x );
 #include "wininet_functions.inc"
This code fragment makes extensive use of the C++ preprocessor to automate the generation of boilerplate code. The SPYMACRO macro uses the token pasting operator (##) and the stringizing operator (#) to create a macro that takes a function name as input and expands to something like:

 g_pfnInternetOpenA =
     GetProcAddress(hModWininet, "InternetOpenA" );
The line that reads #include "wininet_functions.inc" is just a list of the functions exported from WININET.DLL. The contents of this file begin like this:

 SPYMACRO( AuthenticateUser )
 SPYMACRO( CommitUrlCacheEntryA )
The final result of this preprocessor funny business is: for each function listed in WININET_FUNCTIONS.INC, DllMain calls GetProcAddress on that function and assigns the return address to an appropriately named function pointer declared at the global scope. Why go through all the hassle of listing the WININET APIs in a separate file? Why not just include the API list directly in DllMain? Think about all those function pointers that need to be declared at global scope, and therefore outside of the DllMain code. With over 100 WININET APIs, I'd be looking at adding 100 or so additional lines of code to declare these variables.
By putting the API list in a separate file, I can use a different definition for the SPYMACRO macro and #include the "wininet_functions.inc" file a second time. This time, SPYMACRO looks like:

 #define SPYMACRO( x ) FARPROC g_pfn##x;
This expands to something like:

 FARPROC g_pfnInternetOpenA;
Localizing all of the API functions in a separate file has two advantages. First, if I wanted to change the variable names, the call to GetProcAddress, or whatever, I'd make the change in exactly one spot. That is, where the SPYMACRO is declared. Second, if I were to add new WININET API functions to the file, both the function pointer declaration and its corresponding GetProcAddress would automatically appear upon recompiling. (I've read that Bjarne Stroustrup isn't enamored of preprocessors and macros, but I think that they're pretty slick when you can do things like this.)
The remaining code in DllMain is simple code for opening the logging output file and closing it when the process terminates. I'll get to the somewhat unusual output file later. For now let's focus on the code that makes up the majority of WININETSPY.CPP: the API logging stubs.
Take a glance at code for InternetCanonicalizeUrlA (it's the first API stub). As you'd expect, the return value, calling convention, and parameters are exactly the same as the API's prototype in WININET.H. In fact, I simply copied the relevant WININET.H prototypes into WININETSPY.CPP and made functions out of them. The meat of each API logging stub is fairly standard and goes like this:
  • Declare a local variable to store the return value.
  • Call the real WININET API function with the SPYCALL macro.
  • Log the API's name and relevant parameters.
  • Return the value that the real WININET API returned.
The most interesting part of the sequence is the SPYCALL macro. If you've ever used a function pointer returned by GetProcAddress, you know what a pain it can be. In C++, you need to make a typedef corresponding to the function definition, and typecast the return value from GetProcAddress to this typedef:

 typedef INTERNETAPI
     (BOOL WINAPI *PFNINTERNETCLOSEHANDLE)
     (HINTERNET hInternet);
 
 g_pfnInternetCloseHandle = 
     (PFNINTERNETCLOSEHANDLE)GetProcAddress(
         hModWininet, "InternetCloseHandle");
Yuck! What a mess! Now multiply this hassle by over 100 WININET APIs. Alas, it has to be like this so that the compiler can verify parameters and return values for their proper type. The SPYMACRO macro trades off this type safety for a much easier way to invoke the real WININET APIs. Check out the SPYMACRO code near the beginning of WININET.CPP.
The SPYMACRO macro is a sequence of inline assembler instructions that use the two macro parameters: a function pointer to be called and the number of DWORDs passed as arguments to the API. Luckily, the number of DWORDs is usually the same as the number of arguments. The assembler code makes a copy of the API's parameters to a lower location on the stack, and then calls through the function pointer. The function pointer is what transfers control to the real API code in the system-supplied WININET.DLL. Looking through the code, you'll see that the function pointer parameter to SPYMACRO is always one of the g_pfnXXX global variables that I described earlier.
After the real API code returns, the SPYCALL macro cleans the copied parameters off the stack and copies the return value (in EAX) to a local variable. The SPYMACRO code assumes that you've declared a local variable named retValue, a small price to pay for ridding yourself of the compiler's obsessive type checking. All in all, this dancing on the fringe isn't recommended programming practice, but if you're experienced enough to understand the risks and rewards, I say go for it!
After the SPYCALL macro code executes, the logging code in each API stub comes next. To be honest, I didn't bother to log every parameter of every API. Rather, I selected the parameters that were most likely to be informative, strings in particular. Feel free to add to the list of parameters that it logs. To handle the cases where a string parameter might be zero, I wrote the SAFESTR macro. For a given input string, it returns either the same pointer, or a pointer to an empty string ("") if the input string pointer is zero. This let me avoid cluttering up the code with hundreds of checks for valid string pointers.
The last piece of the WININETSPY.CPP to look at is the actual logging code. While the logging occurs via a function called printf, this printf isn't the standard C++ runtime library version of the function. I wrote a replacement printf function (near the top of WININETSPY.CPP) that formats the output like printf would and writes the results to a file. My replacement printf assumes that a global variable named g_hOutputFile has been initialized with a valid file handle. DllMain is where this initialization occurs.
When I first wrote WININETSPY.CPP, my output file was an ordinary disk-based text file. Plain, simple, and easy to work with in an editor. Eventually, I became dissatisfied with disk files. I wanted to see the logging trace as it occurred, and I didn't want to hassle with opening up the output file in an editor. I also wanted to be able to easily throw away all prior logging output and start with a fresh, clean buffer. For example, in figuring out the operations needed to get a fund quote, there are hundreds of uninteresting output lines emitted before getting to the point where you'd click on the Get Quote button. In short, I wanted the logging output to be collected and presented in a different program.
After pondering possible implementations, I hit upon the idea of using the Win32 mailslot facility. Instead of opening a disk file in DllMain, I instead open an existing mailslot and write to its file handle. Nothing else needs to change. I don't have space here to describe mailslots in any detail. The important thing is that I could treat each line of logging output as a message and lob it into the mailslot. The logging display program simply needs to read from the mailslot in a timely manner and display each message.
At this point, Visual Basic entered the picture. In a manner of minutes, I whipped together a Visual Basic program consisting of a form and an edit control where I appended each line of output read from the mailslot. Later, I got fancy and changed the edit control to a rich text control to get cool features like searching and buffers greater than 64KB. I called the finished program WININETSPYMon (see Figure 2).
Figure 2 WININETSPYMon
Figure 2 WININETSPYMon

While I won't go into all the details of WININETSPYMon here, two important procedures merit further commentary. In the Form_Load procedure, the code first creates the mailslot, which it names wininetspymon_mailslot. In the Timer1_Timer procedure, the code calls the GetMailslotInfo and ReadFile APIs in a loop until there are no more remaining messages. Each message (that is, line of output) is appended to the end of the edit control. The Timer1_Timer procedure is called every 50 milliseconds via the standard Visual Basic timer control.
The features of WININETSPYMon are mostly self-evident. To clear the edit control, click the Clear output button. To search for a string in the output, type the search text into the bottom edit control, then click the Find button. You can click Find again to continue the search. The output edit control has the read-only attribute, but you can select and copy text out of it, allowing you to save some or all of the output to a disk-based file. I could have spent more time adding a lot more features, but WININETSPYMon is good enough for its intended purpose. If I'm going to spend time writing Visual Basic code, I want it to focus on more interesting things, like my investment analysis program.
To wrap up, here's a short list of things to keep in mind when setting up and using WininetSpy.
  • Remember to run the WININETSPYMon installation program. This will make sure that you have the required Visual Basic 5.0 runtime library DLL and RICHTX32.OCX installed on your system.
  • Copy the WininetSpy version of WININET.DLL to the same directory as IE. On my system running Windows NT 4.0 this is C:\Program Files\Plus!\ Microsoft Internet.
  • Start up WININETSPYMon before starting up IE. This is necessary because WININETSPYMon creates the mailslot that my WININET.DLL looks for in its DllMain.
  • If IE doesn't work correctly (for example, it's unable to connect to the Web or display pages), the problem may be newer versions of the system DLLs such as URLMON.DLL or MSHTML.DLL. These DLLs may be importing additional functions from WININET.DLL that the logging version doesn't provide. The solution would be to add stubs for the missing functions. Unfortunately, I'm not aware of any reliable and easy-to-use method of determining which API stubs the system DLLs are looking for, but not finding.
  • Don't forget to delete or move the logging WININET.DLL from the IE directory when you're finished spying. There's no sense in slowing down your system or risking funky behavior when you don't need to. For example, I wasted over an hour trying to figure out why I couldn't access a page that needed 128-bit encryption. It turned out an encryption DLL (SCHANNEL.DLL) determined that my WININET.DLL was not the same as the Microsoft version. It therefore refused to do 128-bit encryption because, at the time of this writing, 128-bit encryption is still legally classified as a munition by the U.S. government. Happy spying!

Have a question about programming in Windows? Send it to Matt at mpietrek@tiac.com

From the September 1997 issue of Microsoft Systems Journal. Get it at your local newsstand, or better yet, subscribe.

© 1997 Microsoft Corporation. All rights reserved.
Terms of Use
.

© 2014 Microsoft Corporation. All rights reserved. Contact Us |Terms of Use |Trademarks |Privacy Statement
Microsoft