My favorite test trick is...
On This Page
Don Burn: ...not a trick at all, but an approach to development and test.
My favorite test trick is not a trick at all, but an approach to development and test. Drivers can be complex projects, so many of us develop drivers as a set of layered code modules. The trick here is to build a test environment as you build the layers. To do this you need to build a simple test harness driver for each layer. These test harnesses can use the DriverEntry routine with DbgPrompt and DbgPrint to provide a kernel version of a simple console application. An article describing this in detail is, "Try This -- Interactive Driver Testing." 
The bottom layers of a typical driver are service functions that only depend on the kernel support routines. While coding a layer, develop a test harness to exercise that layer's routines. The test harness driver should be built with the Call Usage Verifier and loaded on a checked build of Windows 2003 with the driver verifier enabled. This provides a powerful debugging environment by using the most comprehensive validity checking. Development of the next layer should start once the test harness runs cleanly and the code is fully exercised (if you have access to code coverage, use it).
When layering above a previously developed and tested module, provide debug output that shows the calls to the lower layer as input to the layer's test harness. As you create the new layer, again create a test harness to exercise the new layer. Obviously, only one test harness runs at a time, but if you do find a problem in a previous layer, dust off that test harness and fix the problem.
Finally, as you develop and maintain the driver, keep each layer's test harness up to date. Building these test harnesses gives you an easy way to exercise and debug the driver in the future.
Thomas Divine: ...to study the DDK Header Files.
I don't think anyone can deny that the DDK documentation has greatly improved over the past few years. Even so, there is one source of DDK documentation that is often overlooked: the DDK header files.
In my limited world of NDIS device drivers, examining the NDIS.H header exposes some information that isn't really "hidden," but may be only briefly mentioned in the DDK documentation. For example, looking at the NDIS.H header provides the following information about an NDIS buffer descriptor:
1. | An NDIS buffer descriptor (NDIS_BUFFER) is actually identical to an NT memory descriptor list (MDL), at least on "Big Windows." This fact is only mentioned in three places in the current DDK, and two of the three references to this aren't actually under NDIS topics at all. |
2. | Many "functions" that manipulate NDIS buffer descriptors are actually MACROs around ordinary NT kernel services. For example, NdisQueryBufferSafe is just a wrapper around MmGetSystemAddressForMdlSafe and MmGetMdlByteCount. |
Other information that can be extracted from NDIS.H includes the relationship between the various "strings" that are used in NDIS drivers:
| • | NDIS_STRING |
| • | UNICODE_STRING |
| • | STRING |
| • | ANSI_STRING |
| • | CSTRING |
| • | OEM_STRING |
Knowing these relationships is essential when it becomes necessary to find an appropriate string manipulation function.
There are tons of "interesting" MACRO definitions, as well as comments from Microsoft developers in the DDK header files.
I am certainly not recommending that you use the DDK headers as a substitute for the DDK documentation in the Help file. In fact, details defined in the DDK header files can change without changing the DDK documentation.
But, I do recommend that some time in the upcoming Holiday season you curl up in front of a cozy fire and do some light reading of the DDK header files. Here's my suggested short reading list for NDIS driver writers:
| • | NTDDK.H |
| • | NTDEF.H |
| • | NDIS.H |
Tim Roberts: ...give the product to someone who wasn't involved in its development.
My favorite test trick is to give the product to someone who wasn't involved in its development. I've been programming computers for 35 years, and doing system-level programming for 30. I dare to consider myself an expert. And yet, even with all of that experience, I still find myself falling into the trap of, "If it hurts when you push it, don't push it." My brain remembers which sequence of things have caused trouble in the past, and it avoids doing those things. When you give the product to someone who doesn't know where it hurts, they push, and push hard. That provides the incentive I need to go in and fix where it hurts.
Mark Roddy: ...actually I have three favorite test tricks, presented here in no particular order:
1. | Open the window. When confronted with a difficult-to-reproduce bug, the temptation is to introduce changes that make the problem even less likely to occur. Unfortunately, Murphy's Law applies, and this obscure bug will be completely and reliably reproducible by your Very Important Customer. Instead of trying to fix the problem by reducing its frequency, try to fix the problem by introducing tests and code changes that increase the frequency. Don't try to close the window, open it wide instead, until you fully understand the root cause of the problem. |
2. | Ceteris Paribus. Sounds good, doesn't it? Say this a lot and people will think you know more than you do. What it means is that if you change just one of many variables, any changes in the behavior of the system under test can be attributed to that one changed variable. It is such a simple concept, and it is a remarkably valuable tool. |
3. | Driver Verifier is your best friend. |
Stephan Wolf: ...disabling the device under heavy load.
A critical situation for every driver is when it unloads. Although most drivers only load once during system startup and then never unload, it is still possible that the user takes some action, which leads to the system unloading a driver. For instance, during a driver update, the system unloads the old driver and then loads the new driver. A user can also disable a device in the Device Manager, which also unloads the respective device driver. Drivers for Plug and Play devices, e.g. USB and PCMCIA devices, usually load and unload on an as needed basis.
Unloading a driver is a simple task on the first view. The driver frees all resources and exits. However, some resources might still be in use if the device or driver is busy. It is therefore crucial that a driver takes appropriate action to properly shut down its device and release all hardware and software resources. This includes interrupts, timers, device and system memory, I/O ports, and so on.
In order to test the robustness of my drivers, I always setup a scenario where I can force the driver to unload when it is very busy. Depending on the type of device/driver, consider the following unload tests:
| • | All devices: Under heavy device load, right-click the device in Device Manager and then click "disable device". |
| • | Network devices (NDIS): Generate as much network traffic as possible. Then unplug the network cable. To generate network load, use a tool like the "standard" TTCP utility (Test TCP) |
| • | USB, PCMCIA, and other Plug and Play devices: Unplug the device during full load. This might not be really "healthy" for all kinds of devices, for example, external hard disk drives (although I would still test this also!). But you can safely unplug printers, scanners, cameras, and so on, while they exchange data with the Windows system. |