Using Spin Locks
A spin lock is a mechanism that kernel-mode driver developers can use to synchronize shared data between multiple threads in Microsoft® Windows®. While one thread owns a spin lock, any other thread that is waiting to acquire the lock "spins" on a memory location. When the spin lock is released, the waiting thread acquires the lock. A waiting thread is not suspended or paged out; it retains control of the CPU, thus preventing execution of other code at the same or a lower IRQL.

Spin locks are opaque objects of type KSPIN_LOCK. They must be allocated from nonpaged memory, such as the device extension of a driver-created device object or nonpaged pool allocated by the caller. In this discussion, we will look at three types of spin locks:

• Ordinary spin locks
• Queued spin locks
• Interrupt spin locks

All types of spin locks raise the IRQL to DISPATCH_LEVEL or higher. Spin locks are the only synchronization mechanism that can be used at IRQL >= DISPATCH_LEVEL. Code that holds a spin lock runs at IRQL >= DISPATCH_LEVEL, which means that the system's thread switching code (the dispatcher) cannot run and, therefore, the current thread cannot be pre-empted. Drivers should hold spin locks for only the minimum required amount of time and eliminate from the locked code path any tasks that do not require locking. Holding a spin lock for an unnecessarily long duration can negatively affects system performance.

IMPORTANT: All code within the spin lock must conform to the guidelines for running at IRQL >= DISPATCH_LEVEL. If code within the spin lock causes a page fault, the result is a system crash with the bug check value IRQL_NOT_LESS_OR_EQUAL. These system crashes occur because at IRQL >= DISPATCH_LEVEL, the operating system cannot wait for the kernel dispatcher event that is set internally when paging I/O completes. For complete information about these guidelines, see the white paper "Thread Context, Scheduling, and IRQL," which is available at http://www.microsoft.com/whdc/hwdev/driver/IRQL.mspx.

To implement spin locks on a single-processor system, the operating system has only to raise the IRQL, which prevents pre-emption of the current thread. Because no other threads can run concurrently, raising the IRQL is adequate to protect any shared structures. (Note, however, that the checked build of the operating system uses spin locks, even on single-processor systems.) On an SMP system, the operating system raises the IRQL and then spins by testing and setting a variable using an interlocked instruction.

Ordinary Spin Locks
Ordinary spin locks work at DISPATCH_LEVEL. To create an ordinary spin lock, a driver allocates a KSPIN_LOCK structure in nonpaged memory and then calls KeInitializeSpinLock to initialize it. Code that runs at IRQL < DISPATCH_LEVEL must acquire and release the lock by calling KeAcquireSpinLock and KeReleaseSpinLock. These routines raise the IRQL before acquiring the lock and then lower the IRQL upon release of the lock.

Code that is already running at IRQL = DISPATCH_LEVEL should call KeAcquireSpinLockAtDpcLevel and KeReleaseSpinLockFromDpcLevel instead. These routines do not change the IRQL.

Queued Spin Locks
Queued spin locks are a more efficient variation of ordinary spin locks. Queued spin locks are available in Microsoft Windows XP and later releases of Windows. Whenever multiple threads request the same queued spin lock, the waiting threads are queued in order of their request. In addition, queued spin locks test and set a variable that is local to the current CPU, so they generate less bus traffic and are more efficient on non-uniform memory architectures (NUMA).

A queued spin lock requires a KLOCK_QUEUE_HANDLE structure in addition to a KSPIN_LOCK structure. The KLOCK_QUEUE_HANDLE structure provides storage for a handle to the queue and the associated lock. This structure can be allocated on the stack. To initialize a queued spin lock, the driver calls KeInitializeSpinLock.

To ensure that the IRQL is properly raised and lowered, driver routines that run at PASSIVE_LEVEL or APC_LEVEL must call KeAcquireInStackQueuedSpinLock and KeReleaseInStackQueuedSpinLock to acquire and release these locks Driver routines that run at DISPATCH_LEVEL should call KeAcquireInStackQueuedSpinLockAtDpcLevel and KeReleaseInStackQueuedSpinLockFromDpcLevel instead. These routines do not raise and lower the IRQL.

Queued Spin Locks
An interrupt spin lock protects data such as device registers that a driver's InterruptService routine and SynchCritSection routine access at the specified device interrupt request level (DIRQL). When a device driver connects its interrupt object, the operating system creates an interrupt spin lock associated with that interrupt object. The driver is not required to allocate storage for the spin lock or to initialize it.

When an interrupt occurs, the system raises the IRQL on the processor to DIRQL for the interrupting device, acquires the default interrupt spin lock associated with the interrupt object, and then calls the driver's InterruptService routine. While the InterruptService routine is running, the processor IRQL remains at DIRQL and the operating system holds the corresponding interrupt spin lock. When the InterruptService routine exits, the operating system releases the lock and lowers the IRQL (unless another interrupt is pending at that level).

The system also acquires the default interrupt spin lock when a driver calls KeSynchronizeExecution to run a SynchCritSection routine. The operating system raises the IRQL to DIRQL for the device, acquires the lock, and invokes the SynchCritSection routine. When the routine exits, the operating system releases the lock and lowers the IRQL. Other driver routines that share data with the InterruptService routine or SynchCritSection routine must call KeAcquireInterruptSpinLock to acquire this lock before they can access the shared data. KeAcquireInterruptSpinLock is available on Windows XP and later releases of Windows.

Some types of devices can generate multiple interrupts at different levels. Examples include devices that support PCI 3.0 MSI-X, which generate message-signaled interrupts (MSI), and a few older devices that interrupt at more than one IRQL. Drivers that support such devices must serialize access to data among two or more InterruptService routines.

In this case, the driver must create a spin lock to protect shared data at the highest DIRQL at which any interrupt may arrive. When the driver connects its interrupt objects, it passes a pointer to the driver-allocated KSPIN_LOCK structure, along with the highest DIRQL at which the device interrupts. The operating system associates the driver-created spin lock and the DIRQL with the interrupt object.

When the operating system calls the InterruptService routines, it raises the IRQL to the DIRQL specified with the interrupt object and acquires the driver-created spin lock. The system also uses this lock when it runs a SynchCritSection routine. Other driver routines that share data with the InterruptService or SynchCritSection routine must call KeAcquireInterruptSpinLock to acquire this lock before they can access the shared data.

The next version of Microsoft Windows (codenamed "Longhorn") includes significant changes to the interrupt architecture to support message-signaled interrupts. Specifically, the IoConnectInterrupt routine is deprecated. You should use its replacement, IoConnectInterruptEx in new drivers, and older versions of drivers should be updated to use this new routine if possible.

For more information on these changes, see the white paper "Interrupt Architecture Enhancements in Microsoft Windows Codenamed Longhorn" at:
http://www.microsoft.com/whdc/hwdev/bus/pci/msi.mspx

The preview (pre-beta) version of the Windows Longhorn Driver Kit (LDK) provides the IoConnectInterruptEx routine for use in Microsoft Windows 2000 and later releases of Windows.

To find out more about obtaining a copy of the LDK, see:
http://www.microsoft.com/whdc/hwdev/longhorn/default.mspx

Call to Action
Nearly every driver requires spin locks and these mechanisms by their very nature can cause performance bottlenecks. Follow these guidelines to improve performance:

• Test your Windows kernel-mode driver using the Driver Verifier tool (verifier.exe) in Microsoft Windows XP and later operating systems. Use Driver Verifier global counters to monitor IRQL raises and spin lock acquisitions.
• Use an in-stack queued spin lock instead of an ordinary spin lock whenever several components might frequently contend for the lock
• Hold each spin lock for the minimum amount of time necessary, particularly if the lock is frequently required by other code. For example, traversing a long, linked list in a linear order while holding a heavily used spin lock can cause a performance bottleneck.
• For more information about IRQL issues for drivers, see the white paper "Scheduling, Thread Context, and IRQL," at http://www.microsoft.com/whdc/hwdev/driver/ IRQL.mspx
• Additional information and best practices for driver developers concerning spin locks, fast mutexes, InterlockedXxx and ExInterlockedXxx routines, and other synchronization mechanisms in Windows kernel-mode drivers can be found in the "Locks, Deadlocks, and Synchronization" white paper at http://www.microsoft.com/whdc/hwdev/driver/ locks.mspx

Microsoft iSCSI Software Initiator Version 1.03
An updated version of the Microsoft iSCSI Software Initiator package (version 1.03) is now available. The Microsoft iSCSI Software Initiator allows a Microsoft Windows-based computer to serve as an iSCSI initiator to connect to iSCSI targets on an Internet Protocol Storage Area Network (IP SAN). All iSCSI devices appear in Windows as a local disk and can be managed in Disk Administrator, as any other local disk.

Supported Operating Systems:

• Windows Server^TM 2003
• Windows XP Service Pack 1
• Windows XP 64-Bit Edition
• Windows 2000 Service Pack 3 or later

The following fixes were made in Microsoft iSCSI Software Initiator Version 1.03 release.

• On SCSI command timeout or on BUS Reset initiator will send Task Management command and will reset the TCP connection. It will then re-login to the session.
• A PDU with invalid command sequence number was being sent upon bus reset.
• An invalid response for SCSIOP_REPORT_LUNS was causing the initiator to crash.
• A Task Management command was being sent with an invalid logical unit number (LUN).
• An invalid header was being sent in response to R2T request.
• Tag leak caused under low memory condition.
• Initiator does not assume that padding bytes in a PDU will be zero. It will use whatever data is in the padding bytes to compute CRC.
• Handle IOCTL_SCSI_GET_CAPABILITIES and IOCTL_SCSI_GET_INQUIRY_DATA by PDO in Windows 2000.

Call to Action
Download the appropriate Microsoft iSCSI v1.03 package for your operating system at:
http://www.microsoft.com/downloads/details.aspx? FamilyID=12cb3c1a-15d6-4585-b385-befd1319f825&DisplayLang=en

For more information on usage of the Microsoft iSCSI Software Initiator v1.03, please see the user's guide included with the package.

Come Together at WinHEC 2004
WinHEC is the single opportunity each year for the industry to work together on critical technology issues and future directions for Windows, and to interact with Microsoft engineers and other industry leaders.

At WinHEC 2004 in Seattle, Microsoft Chairman and Chief Software Architect Bill Gates will present the opening keynote about Microsoft investments and industry opportunities for the Microsoft Windows platform.

Register before March 22 and save $300!

For more information on the conference and instructions for registering, see:
http://www.microsoft.com/whdc/winhec

The Microsoft Hardware Newsletter, a publication of Microsoft Windows Hardware and Driver Central, is a resource for developers who are creating new hardware and drivers for Microsoft Windows operating systems.

Did you get this newsletter from a friend? To subscribe to the Microsoft Hardware Newsletter, visit: http://register.microsoft.com/regsys/decisionpoint.asp.