PC Storage Directions

The evolution of hard disk and optical solutions

Updated: December 4, 2001

User demand for PC storage capacity continues to grow, especially for large audio, video, image, and multimedia files. This capacity demand affects hard disks and removable media such as CDs and DVDs.

PC storage technologies are evolving to meet user demands for increasingly greater capacity and more flexible capabilities.

The Microsoft Windows XP operating system delivers new support for storage devices, including performance optimizations, support for the largest hard disks and for storing data on optical disks, plus the ability to boot the PC from new kinds of devices on new connection technologies.

Consumer Storage Options

This article focuses on storage options for consumer PCs in general, and for PCs that will run Windows XP Home Edition or Windows XP Professional.

Common PC storage options support a variety of applications and capacities:

•	Hard disk for PC data storage. Current disk size limits of 130 GB will be surpassed with the upcoming availability of devices supporting new ATA/ATAPI-6 48-bit logical block addressing (LBA) technology. This article focuses on ATA as the connector for the host controller on client PCs, with some discussion of IEEE 1394 storage.
•	CD optical devices for PC data and audio storage. These are typically used for software installation and audio recording and playback. Current size limits are 650 MB of data or 74 minutes of audio. CD-RW optical devices are becoming increasingly popular with users because of their ability to hold much more data than floppy disks or other removable storage media.
•	DVD for video playback, and storage of audio, video, and PC data. These discs hold 4.7 GB to 17 GB of data, or more than a full-length feature movie. Users are increasingly choosing DVD drives as a PC option because DVD was designed for multimedia applications such as video playback. This article describes new support and background information about various DVD formats. However, it does not discuss DVD device design and the graphics subsystem support required for a PC that will support a good video playback experience.
•	Flash memory devices. PC Card and other miniature form factors have been increasingly popular with mobile PC users. There is a growth of consumer interest in digital cameras, for which Microsoft has provided new easy transfer, editing, printing, and Web publishing support in Windows XP. We expect to see a growing consumer interest in being able to transfer images from the new proprietary miniature storage devices to the PC. This article does not describe this type of storage. Microsoft expects to see USB implemented as the popular interconnect for PC devices that can read such media.

Top of page

Issues for PC Storage

Ease of Use Issues. New PC storage devices can support a variety of user applications. Both Windows developers and device designers have to address issues such as the following:

•

Removable media drives and new Windows boot capabilities. BIOS issues, unique removable media identifiers, dynamic configuration changes on hot-pluggable buses such as USB or IEEE 1394all create technical implementation problems, and make it necessary that system designers thoroughly understand Windows built-in support. In addition, ease-of-use issues must be addressed to both protect data and allow flexibility for the user.

•

Floppy drive replacement. The limited storage capacity of floppy disks compared to other removable media has made the floppy drive a relic of the past. Cost-effective, easy-to-use, read/write optical media devices can solve the problem of writing large amounts of data to a single disk. However, new issues of compatibility arise, if the resulting disc cannot play in other CD or DVD drives.

New Boot Models. Windows XP delivers boot support for a variety of new media and connection options. Giving users full flexibility for using different boot storage devices requires correct, standards-based firmware support for:

•	Boot order/drive selection in the BIOS
•	Boot support for ATAPI, CD, DVD, and IEEE 1394 devices

Performance Improvements. In particular, to realize the fast system startup/fast resume goals for Windows XPespecially in home scenarios where quiet operation and instant access are importantimprovements in hard disk performance are required. This also requires BIOS refinements and support for bus mastering.

Compatibility and Installed Base. Most PCs shipped today include a high-capacity removable media drive, typically a CD-ROM or CD-RW drive. Lower cost has been a primary reason, to date, for the popularity of CD technology over DVD and other removable media. However, users expect continued support from PCs and from Windows for the full range of current storage options.

One crucial element of this support is selection of the file system for devices. For compatibility and performance reasons, Microsoft believes that this comes down to these decisions for devices that will share data with PCs:

•	NTFS for the hard disk file system
•	FAT or FAT32 for removable media, including flash-type memory
•	CDFS (ISO-9660) for CD-R or CD-RW media

Top of page

Windows XP and Storage

New and enhanced support in Windows XP focuses on these critical areas:

•	Improvements for system boot and resume processes, including functions that reduce and optimize disk reads.
•	Native support for CD-R/RW through a new Imaging Mastering API (IMAPI), to make copying files to CD as easy as it is to copy files to a floppy disk.
•	New support for DVD-RAM.
•	Enhanced support for multi-function MMC-compliant drives.
•	Solid support for ATA/ATAPI controllers and devices, including support for 100-MHz ATA hard disks.
•	Support for system manufacturers to select cycle times, to enable faster Ultra DMA (UDMA) support.
•	Support for removable media identification. Based on proposals to the T13 and T10/MMC working groups, this support enables both software- and user-based content protection schemes.

Windows XP also includes support for IEEE 1394-connected hard disks, as was implemented in Windows Millennium Edition (Windows Me). Driver support for IEEE 1394 hard disks is based on the Serial Bus Protocol-2 (SBP-2) specification.

Planned Additions. Support will be implemented in Windows XP for the new ATA/ATAPI-6 48-bit LBA technology when this technology reaches market. This technology is intended to overcome current 130-GB size limitations for ATA drives. Microsoft also plans to expand IMAPI to include DVD technologies.

Storage Device Drivers in the Windows DDK

http://www.microsoft.com/whdc/devtools/ddk/default.mspx

Boot Support for Windows XP

Under Windows XP, it is possible to boot the system from hard drives connected to ATA, SCSI, Fibre Channel, and the IEEE 1394 bus. (Note, however, that IEEE 1394 support does not include support for dynamic disks.)

It is also possible to boot and install the operating system from CD or DVD devices connected to ATAPI, SCSI, USB, and IEEE 1394 buses, assuming that the device and the BIOS comply with the El Torito specification.

Installation of Windows XP from the retail CD product is limited to device classes supported during the first portion of setup and for which the system BIOS includes Int13 support. PCs that supply CD class drive support by way of PCMCIA or CardBus connections must use "recovery" CD or DVD images to reinstall the factory operating system image.

Note, however, that only the Microsoft storage stacks for ATA and SCSI are guaranteed to support hibernation and crash dump. There is no support for these capabilities for storage connected by way of USB or IEEE 1394.

NTFS for Hard Disks

NTFS is the preferred file system for systems running Windows XP because of the recoverability, scalability, functionality, and enhanced performance benefits that NTFS provides for users.

Microsoft encourages system manufacturers to provide NTFS volumes on all systems where Windows XP is preinstalled in order to gain benefits related to recoverability, scalability, functionality, and performance on large volumes.

Recoverability. NTFS is a journaling file system. As NTFS makes changes to the on-disk format, it writes a log of the changes being made, similar to the technique used by a database to capture transactions for rollback. This journaling feature is important when a system crashes, loses power, or is reset unexpectedly.

In the event of a system reset or crash, NTFS is able to quickly return the disk to a consistent state without running Chkdsk. This robustness yields a better user experience, ensuring fewer reasons for users to make support calls.

In contrast, during a system reset or crash, the FAT file system must always run a Chkdsk process before it mounts the volume. This recovers the FAT file system, but provides a poor user experience because FAT Chkdsk can take a long time to complete, and it can result in support calls.

Scalability. NTFS is designed to scale to very large disk sizes, making it possible to partition large disks with a single volume. The theoretical practical maximum size for an NTFS volume is 2 terabytes. In the next few years, hard disk sizes are expected to be 64 GB to 256 GB in size, even on consumer systems. In addition:

•	NTFS can support large volume sizes with no real constraints on cluster or disk size (see Size Constraints for File Systems table). By comparison, the FAT file system has constraints that affect customer scenarios.

Disk size	Required cluster size for a FAT32 volume
16 GB	4K
64 GB	16K
128 GB	32K (the largest cluster size Windows 95/98 tools can handle)
256 GB	64K (not supported by Windows 95/98/Me)

•

NTFS supports much larger file sizes than FAT32, making it more suitable for large data files such as streaming media. For example, the 4 GB file size limit on a FAT volume is sufficient for only 18 minutes of downloaded video from a digital-video camcorder, and a FAT volume cannot contain a full-length MPEG-encoded movie.

NTFS functionality. NTFS offers advanced features that are either unavailable or difficult to implement on FAT32. Some of these include:

•	Security and access control on private files
•	Distributed link tracking (Windows shell shortcuts will track files even if they are moved)
•	File compression and file encryption
•	Snapshot Backup (enables backup of open files)

For information about the advanced features of NTFS, see "Enterprise Class Storage in Windows 2000" at http://www.microsoft.com/ windows2000/ techinfo/ howitworks/ fileandprint/ storage.asp

Notice that applications sometimes offer additional or improved functionality on NTFS volumes. For example, when burning audio CDs, Windows Media™ Player 8 uses hard links, so that it uses disk space twice as efficiently on NTFS compared to FAT file systems.

NTFS performance with large volumes. NTFS performs well when reading, writing, and mounting large volume sizes.

FAT32 performance is reduced for volumes larger than 32 GB in two areas:

•	Boot time is increased due to time required to read all of the FAT structure to calculate the amount of free space.
•	Read/write performance is affected because the FAT file system must determine the free space on the disk through the small views of the massive FAT structure.

High-performance file system configurations for end-users. Compared to Windows 2000, NTFS has gained numerous performance improvements in Windows XP, plus improved disk defragmentation. Microsoft will continue to optimize NTFS performance.

Size constraints for file systems
	FAT16	FAT32	NTFS
Maximum file size
	4.294 GB	4.294 GB	16.384 terabytes
Minimum volume size
	4085 clusters	65535 clusters	1 MB
Maximum volume size
	2 GB 65,524 clusters	Windows 2000/Windows XP: Can format to 32 GB; can mount/convert larger volumes Windows Me: Up to 268,435,444 clusters Windows 95/98: 4,177,918 clusters	4,294,967,296 clusters
Files per volume
	65,536	268,435,456	4,294,967,295
Directory size
	65534 physical directory entries; special limits on root directory	65534 physical directory entries	No limit

Cluster size - for all file systems: Windows XP, Windows 2000, Windows NT:

•	A power of 2 between 512 bytes and 65536 bytes, inclusive Windows 95/98, Windows Me:
•	A power of 2 between 512 bytes and 32768 bytes, inclusive

CD-MRW and Windows XP

In addition to support for CD media creation with IMAPI (described later), Microsoft expects to provide support for the Mt. Rainier CD-MRW physical format, once drives that implement this technology reach the market.

Together, the Mt. Rainier standards and the IMAPI model will make CD-MRW the best replacement for the floppy drive. Support for CD-MRW drives may be included in Windows XP and all future versions of Windows after these drives reach market, depending on market penetration.

IMAPI and Windows XP

IMAPI allows the user to write files to a CD-R/RW drive by repeatedly dragging-and-dropping files until the disc is full.

IMAPI allows an application to "stage" and "burn" a simple audio or data image onto a 74-minute CD on a set of CD-R or CD-R/W devices. IMAPI will provide an interface between the Windows shell and Windows Media Player.

IMAPI limits

Recording	- Track-at-Once recording - Only 74-minute CD support
Formats	- Redbook audio - Joliet data discs
Space used on disc	- ~700 MB for full disc-master image, created during the mastering process, plus the size of files in staging area. - For NTFS volumes where the information is on the local volume, only hard links are placed in the staging area. - For FAT volumes or files on a different NTFS volume, complete copies are placed in the staging area.

After the user has selected all the files to be burned onto a disc in one session, the next act is to click the Write to CD button to name the disc and start the mastering/writing process. The user can add more files later, but there is a ~50 MB overhead associated with each additional session.

From the Windows Media Player interface, discs can be created that can be read in most consumer players. This requires that the discs must be written in a single session; otherwise, the disc cannot be read by consumer players.

Microsoft developed IMAPI in partnership with Roxio, a subsidiary of Adaptec, Inc. (http://www.roxio.com )

IMAPI and Third-Party Vendors. IMAPI support in Windows means that OEMs can offer basic CD recording support without having to include a third-party application. The built-in support makes CD burning an easy, good experience for their users.

The IMAPI interface will be published shortly after Windows XP Beta 2 in the Microsoft Platform SDK. IMAPI is an open interface that supports third parties adding their own user interface.

IMAPI SDK

http://www.microsoft.com/ msdownload/ platformsdk/ setuplauncher.asp

To take advantage of IMAPI, vendors must ensure that their firmware passes the new MMC compliance test included in the Windows XP HCT from WHQL. This test checks for:

GET_CONFIGURATION
GET_EVENT_STATUS_NOTIFICATION

Details about this test are provided in the Windows HCT 10.0.

Windows XP HCT and Test Specifications

http://www.microsoft.com/whdc/whql/default.mspx

Top of page

Windows and DVD

Microsoft focuses its DVD technology support on compliance with industry standards and the formats that have received high market acceptance by OEMs. Windows XP and future versions will deliver industry-standard methods to support multiple DVD formats.

New DVD support in Windows XP includes support for DVD-RAM (read/write), in addition to support for DVD-Video and DVD-ROM, the read-only versions supported in earlier versions of Windows.

DVD formats
Several different formats are currently defined for DVD.

	Double-side capacity	Read/write support/ sequence	Commercial availability
DVD-Video (video playback) DVD-ROM (data storage only)	17 GB	Read only	Now
DVD-R (data storage)	4.7 GB	Read and one-time write	Now
DVD-RAM (video playback, data storage)	9.4 GB	Read and up to 100,000 rewrites; random	Now
DVD-RW; DVD+RW (video playback, data storage)	9.4 GB	DVD-RW: Read and up to 1,000 rewrites; sequential DVD+RW: Read and rewrites; random	Fall 2001

DVD-RAM support in Windows XP uses the FAT32 file system for read/write and allows use of the Universal Disk Format (UDF) file system for read-only. DVD-RAM also supports multisession recording or logical block addressing.

Microsoft delivers many built-in capabilities to support DVD playback of video, including DirectX video acceleration, DirectShow filters, and new Video Mixer Rendering support in Windows XP. These capabilities are not discussed in this article. For details, see http://www.microsoft.com/whdc/device/stream/dvd/default.mspx

DVD feature summary
Feature	Description
DVD device	- Bus mastering and UDMA/DMA - Media Status Event Notification as defined in MMC-2 standard - Multisession capabilities per OSTA MultiRead Specification for CD-ROM, CD-R, CD-R/RW, and DVD-ROM Devices - 2 MB per second minimum transfer rate or better performance anywhere on the disc - Logical and physical CD formats, MMC-2, and defect management - Correctly displays content with multiple aspect ratios, using information in the MPEG header as the default
DVD video decoder	- Driver correctly handles media types, time discontinuity, and decode-rate adjustment - Line21 closed-caption data
DVD decoder	- Subpicture compositing and closed captioning - Alpha blending of subpicture content on static menus
- MPEG-2 decoder	- Accepts an MPEG-2 elementary stream - MPEG decoder with motion compensation or hardware acceleration uses the Microsoft-provided DirectX VA API
Subpicture decoder	-Correct handling of subpicture properties and other functions
DVD video player	-Navigates chapter breaks seamlessly

DVD Support in Other Windows Versions. Microsoft Windows 2000, Windows Me, and Windows 98 Second Edition support the following capabilities for DVD drives:

•	DVD movie playback
•	DVD as a read-only data storage device (DVD-ROM)
•	DVD-ROM class driver and UDF read-only file system for data storage
•	Streaming class driver
•	System boot from DVD drives that comply with the El Torito specification
•	Microsoft DirectShow and DirectDraw hardware acceleration support, with Video Port Extensions (VPE) for DVD playback

Top of page

Windows Logo Program Requirements

Most standards and design guidelines for storage and controllers are well defined in the industry. The Windows Logo requirements reflect those standards as well as incremental advances made in the industry and at Microsoft. For Windows XP, new requirements include:

•

Controller and devices must support bus mastering.

UDMA/DMA Programmed I/O (PIO) support is allowed only for flash memory devices.

To tolerate UDMA, ATAPI devices must support the termination scheme defined in ATA/ATAPI-5.

•

Hard drives must support ATA/ATAPI-5, minimum. Support for ATA/ATAPI-6 will be a requirement for 48-bit LBA and ATA Removable Media Serial numbers.

•

CD and DVD drives must support the ANSI T10 MMC-2 standard plus features and commands intended to support the built-in features of Windows XP.

Windows Logo Program Requirements

http://www.microsoft.com/whdc/winlogo/default.mspx

Top of page

Guidelines for Manufacturers

For device manufacturers:

•	Do not invent a proprietary file system. Functional and useful file systems exist todayconsider FAT32.
•	Send Microsoft your prototypes and currently shipping hardware for compatibility testing in the Windows development labs. Work with us in early phases to identify issues and ensure optimal design.
•	Follow industry standards and test with the HCTs from WHQL to ensure compatibility with Windows.
•	For USB devices, always populate the device serial and model numbers, even though a specification may define such identifiers as optional.
•	For writable optical devices, implement IMAPI support for the device and any utilities you create.
•	Use built-in Windows driver support and follow the Windows DDK when building miniport drivers for storage.

Windows DDK http://www.microsoft.com/whdc/devtools/ddk/default.mspx

For system manufacturers:

•	Choose compatible devices, based on the Windows Logo Program.
•	Choose devices whose performance supports the goals for fast startup/fast resume. Follow the guidelines and test with the tools: http://www.microsoft.com/ whdc/ hwdev/ platform/ performance/ fastboot/default.mspx
•	Format hard disks as NTFS in your factory preinstallation process. See the OPK for tools and details.
•	Select IMAPI-based solutions for read/write optical devices, to ensure best experience for end users and lower support costs.
•	Do not implement multiple logical unit numbers (LUNs) optical drives. See: http://www.microsoft.com/ whdc/ hwdev/ tech/ stream/ dvd/ multiLUN.mspx
•	Work with your BIOS vendor to ensure that the system firmware adequately addresses issues for multiple types of boot storage devices, fast-resume performance, and so on.
•	Send Microsoft your prototypes and shipping hardware, for compatibility testing in the Windows development labs. In particular, make sure that Microsoft is testing any new type of hardware or solution that you are testing.

Resources from Microsofthttp://www.microsoft.com/whdc/device/storage/default.mspx

dvd@microsoft.com

Top of page

Industry Standards and Activity

The ANSI NCITS T10 and T13 working groups are the primary standards bodies for PC storage devices. The T10 group publishes the MMC standards for CD and DVD drives, and specifications for the SCSI architecture. The T13 group publishes standards for ATA and ATAPI interfaces.

ANSI T10 Working Group

http://www.t10.org

The Mt. Rainier Group, a joint effort of Microsoft, Compaq, Philips, and Sony, produced the CD-MRW protocol, which has been submitted to the T10/MMC.

Mt. Rainier Group

http://www.mt-rainier.org

Top of page