*
MSDN*
Results by Bing
|Developer Centers|Library|Downloads|Code Center|Subscriptions|MSDN Worldwide
Search for


Advanced Search
MSDN Home > MSJ > May 1998
May 1998

Microsoft Systems Journal Homepage

Minimizing the Memory Footprint of Your Windows CE-based Program

Douglas Boling

A Windows CE machine may have only 1 or 2MB of RAM, which is tiny. Fortunately, the functions available for managing that memory are fairly complete. Windows CE implements almost the full Win32 memory management API that is available under Windows NT and Windows 98.

This article assumes you're familiar with Win32 and C

Code for this article: WinCEMemory.exe (63KB)
Douglas Boling is an electrical engineer and writer who has worked with PCs since the mid 70's. He is also a contributing editor to Microsoft Interactive Developer and PC Magazine. He is currently writing Programming Windows CE for Microsoft Press.

Each major version of Windows® has a different mission. Windows 98 and Windows NT® are targeted to the desktop and corporate server markets. Windows CE, on the other hand, is designed for small platforms. Because of the size and capabilities of the hardware that runs it, Windows CE and applications written for it must be small and extremely memory efficient. A Windows CE machine may have only 1 or 2MB of RAM, which is tiny compared to a typical personal computer that has between 16 and 64MB of RAM. In fact, memory on a Windows CE machine is so scarce that it is often necessary to write programs that conserve memory even to the point of sacrificing the overall responsiveness of the application.
Fortunately, while the amount of memory may be small in a Windows CE system, the functions available for managing that memory are fairly complete. Windows CE implements almost the full Win32
® memory management API available under Windows NT and Windows 98. Windows CE supports virtual memory allocations, local and separate heaps, and even memory-mapped files.
Like Windows NT, Windows CE supports a 32-bit flat address space with memory protection between applications. However, since Windows CE was designed for different environments, its underlying memory architecture is different from Windows NT. These differences can affect how you design a Windows CE-based application. In this article, I will cover the basic memory architecture of Windows CE. I will also cover the different types of memory allocation available to Windows CE-based programs and how to use each memory type to minimize your application's memory footprint.

Basics

The RAM in a Windows CE-based system is divided into two areas: the object store and the program memory. The object store is similar to a permanent, virtual RAM disk. Unlike the old virtual RAM disks on a PC, the object store retains the files stored in it even if the system is suspended. This is why Windows CE systems have a backup battery; when you replace the main batteries, the backup battery provides power to the RAM to retain the files in the object store. Even when the user hits the reset button, as the Windows CE kernel starts up, it looks for a previously created object store in RAM and uses it if one is found.
The other half of the RAM is devoted to program memory. Program memory is used like the RAM in personal computers; it stores the heaps and stacks for the applications that are running. The boundary between the object store and the program RAM is movable. The user can move the dividing line between object store and program RAM by using the System Control Panel applet if the platform provided this option. Under low memory conditions, the system will ask the user for permission to take some object store RAM for use as program RAM to satisfy an application's RAM needs.
In a Windows CE-based system, the ROM stores the entire operating system, as well as the applications that come with the system. In this sense, the ROM in a Windows CE system is like a small, read-only hard disk. Under Windows CE, ROM-based programs are executed directly from the ROM instead of being loaded into program RAM and then executed. This is a huge advantage for small systems in two ways. First, the fact that code executes directly from ROM means that the program code does not take up valuable program RAM. Second, since the program does not have to be copied into RAM before it is launched, the time needed to start an application is reduced. Programs that are not in ROM but are contained in the object store or on a flash-memory storage card are not executed in place; they are paged into the RAM and then executed.

Virtual Memory

Windows CE implements a paged virtual memory management system similar to the other Win32-based operating systems (Windows NT and Windows 98). Under Windows CE, a page is either 1,024 or 4,096 bytes depending on the microprocessor, with the 1KB page size preferred by the Windows CE architects. This is a change from Windows NT where page sizes are 4,096 bytes for Intel microprocessors and 8,192 bytes for the DEC Alpha. For the CPUs currently supported by Windows CE, the NEC 4100 series and the Hitachi SH3 use 1,024-byte pages while the Intel and AMD 486, the Phillips 3910, and the Motorola Power PC 821 use 4,096-byte pages.
Virtual pages can be in one of three states: free, reserved, and committed. A free page is, as it sounds, free and available to be allocated. A reserved page has been set aside so that its virtual address cannot be allocated by the OS or another thread in the process. A reserved page cannot be used elsewhere, but it also can't be used by the application because it is not mapped to physical memory. To do that, a page must be committed. A committed page is one that has been reserved by an application and has been mapped directly to a physical address.
All of this discussion should be old hat to experienced Win32-based programmers. The important thing for the Windows CE-based programmer is to learn how Windows CE changes the equation. While Windows CE implements most of the same memory API set as its bigger Win32 cousins, the underlying architecture of Windows CE does impact programs. To better understand how the API is effected, it helps to look at how Windows CE uses memory behind the scenes.

The Windows CE Address Space

In OS circles, much is made of how far the operating system goes to protect one application's memory from other application's memory. Of the 4GB space, 2GB is reserved for hardware access by the OS. The other 2GB is a virtual address space shared across all applications. However, an application's address space is protected from access by other applications through the protection attributes of the individual virtual pages. A diagram of the Windows CE virtual address space is shown in Figure 1. A little over half of the address space is divided into 33 32MB "slots." Each slot is assigned to a currently running process with the lowest one, slot 0, assigned to the active process. As Windows CE switches between processes, it remaps the address space to move the old process out of slot 0 and the new process into slot 0.
Figure 1 Virtual Address Space
Figure 1 Virtual Address Space


The region of the address space above the 33 slots starting at 0x4200 0000 is used to map memory-mapped files. A tiny amount at the very top of the address space is reserved by the system. The memory-mapped area is a shared and unprotected resource. That is, if one application has a file mapped at 0x4200 0000, all other applications in the system have the same access rights to that space. While this is true today, don't count on this characteristic in future versions of the OS. It may change.
Like Windows NT, Windows CE also excludes any process from accessing the lowest 64KB block of the address space. When you think about it, this is required of any Win32 OS since some of the Win32 calls look at the upper word of a parameter to determine if it is a pointer or contains data. Reserving this area also helps catch those annoying uninitialized pointers.
To help determine the global state of the system, Windows CE implements the Win32 GetSystemInfo and GlobalMemoryStatus functions. The information returned by GlobalMemoryStatus provides confirmation of the memory architecture of Windows CE. Figure 2 shows MemDemo, an example program that will perform various feats for this article. The MemDemo window displays the system information using GetSystemInfo, GlobalMemoryStatus, and GetStoreInformation. GetStoreInformation is a unique Windows CE call that returns the state of the object store.

Figure 2 MemDemo
Figure 2 MemDemo


Looking at the information in Figure 2, the dwTotalPhys field indicates that, of the 16MB of RAM in the system, I dedicated 8.2MB of RAM to the program RAM and more than 7MB of that RAM is still free. Note that there is no way for an application using this call to know that another 8MB of RAM has been dedicated to the object store. To determine the amount of RAM dedicated to the object store, an application can call the Windows CE GetStoreInformation function, which returns the amount of RAM allocated for the object store as well as the amount free.
The dwTotalPageFile and dwAvailPageFile fields are zero, indicating no support for a paging file under Windows CE. The dwTotalVirtual field is interesting because it shows the 32MB limit on virtual memory that Windows CE enforces on an application. The dwAvailVirtual field indicates that little of that 32MB of virtual memory is being used in this application.

An Application's Address Space

While it is always interesting to look at the global memory map for an OS, an application should only be interested in its own memory space. Under Windows CE, an application is limited to the virtual memory space available in its 32MB slot. While 32MB may seem like a fair amount of space for an application that may be running on a system with only 4MB of RAM, Win32-based application programmers who are used to a 2GB virtual address space need to keep in mind the limited space available to a Windows CE-based application. An application can use memory-mapped files if huge amounts of virtual memory space are an absolute requirement.
Figure 3 shows a dump of MemDemo's own 32MB virtual address space. Each line of the figure represents a block of virtual memory made up of one or more pages. The addresses of the blocks are offsets into the application's slot in the system address space. The page status is free, reserved, image, or private. Free and reserved were explained previously. Image indicates pages that have been committed and mapped to the image of an executable file in ROM or RAM. Private simply means the pages have been committed for use by the application. The size field indicates the size of the block, which is always a multiple of the page size. The access rights field displays the access rights for the block.
This memory map was captured on a Casio Handheld PC that has an SH3 processor with a 1,024-byte page size. The application used in this example was stored in the object store, then launched. This allowed Windows CE to page parts of the EXE image into RAM only as needed.
The application is mapped at a 64KB region starting at 0x10000. The image of the file contains the code along with the static data segments and the resource segments. It appears that the program code is broken into a number of disjointed pages from 0x10000 to 0x15400. Actually, only the pages containing code that has been executed are mapped into the address space. The reserved pages within the code segment will be mapped into the space only when they are executed.
The read-only static data segment is mapped at 0x18000 and takes three pages. The read-write static data is mapped from 0x19000 to 0x1B3FF. Like the code, the read-write data segment is only committed to RAM as it is written to by the application. Any static data that was initialized by the loader is already committed, and has the static variables written before the address space was captured. The resources for the application are mapped starting at 0x1D000. The resources are read-only, but are only paged into RAM as they are accessed by the application.
The application's stack is mapped starting at 0x20000. The stack segment is recognized easily because the committed pages are at the end of the reserved section, indicating a stack that grows from higher addresses down. If this application has more than one thread, there would be more than one stack segment reserved in the application's address space.
Following the stack is the local heap. The heap has only a few blocks allocated currently, requiring only one page of RAM. The loader reserves another 392,192 bytes (383 pages) for the heap to grow. The over 30MB of address space from the end of the reserved pages for the local heap to the start of the DLLs mapped into the address space is free to be reserved and, if RAM permits, committed by the application.
This application accesses two dynamic link libraries. Coredll.dll contains most of the entry points to the Windows CE operating system. In Windows CE, the API entry points are combined into one DLL—unlike Windows NT or Windows 98 where the core functions are distributed across the Kernel, User, and GDI DLLs. The other linked DLL is the common control DLL, commctrl.dll. As with the executable image, these DLLs are mapped into the address space as linear images. However, unlike the EXE, these DLLs are in ROM and mapped directly into the virtual address space of the application. Therefore, they do not take up any RAM.

Different Methods of Memory Allocation

A Windows CE-based application has a number of different methods for allocating memory. At the bottom of the memory management food chain are the VirtualXxx functions that directly reserve, commit, and free virtual memory pages. Next come the heap APIs. Heaps come in two flavors: the default local heap allocated automatically when an application is started and separate heaps that can be created manually by the application. There is also static data, data blocks defined by the compiler that are allocated automatically by the loader. Finally, there are the stack and memory-mapped files.
One area in the Win32 memory API that Windows CE does not support is the global heap. API calls such as GlobalAlloc, GlobalFree, and GlobalRealloc are thus not present in Windows CE. The global heap is really just a holdover from the Win16 days of Windows 3.x. In Win32, the global and local heaps are quite similar. One unique use of global memory, allocating memory for data in the clipboard, is handled by using the local heap under Windows CE.
The key to minimizing memory use in Windows CE is choosing the proper memory allocation strategy that matches the memory use patterns for a given block of memory. I will review each of these memory types, then discuss strategies for minimizing memory use in Windows CE-based applications.

Virtual Memory

Windows CE supports a subset of the virtual memory functions available under Win32. The main differences are the lack of the VirtualXxxEx functions that manipulate virtual memory in other processes. Windows CE also does not support the VirtualLock and VirtualUnlock functions, since there is no paging file support. However, since the system can discard read-only pages and this can cause problems for some modules such as device drivers, there is a LockPages API provided for OEM use.
Windows CE does support a very handy new page type, the autocommit page. Autocommit pages are initially reserved, but are committed by the system automatically when the page is first accessed. Under Windows CE, there is no need to commit a large region of virtual memory, then access it later. Nor does an application have to reserve a block, then use try/except blocks to commit a page as it is accessed. With autocommit regions, an application simply reserves the region, then writes to it when it needs to. The system takes care of the rest. To allocate an autocommit page or region with this attribute, simply use the MEM_ AUTO_COMMIT flag in the VirtualAlloc call:

 pMem = VirtualAlloc (NULL, 0x4000,
                      MEM_RESERVE | MEM_AUTO_COMMIT,
                      PAGE_READWRITE);

Of course, you should be careful depending too much on autocommitted pages. If there is no RAM left to autocommit, then you will fault. While try/except can catch this, there is nothing you can do to change it. The only way to guarantee that a page will be physically backed up by RAM is to commit the page manually before you need it. Otherwise, Murphy will insure that at the time you access an autocommit page, the system will be too low on memory to satisfy the request.
Before I finish with the virtual memory API, a somewhat subtle point should be made about a Windows CE-based application's virtual memory space. As in Windows NT, virtual memory is reserved in regions that align on 64KB boundaries. Pages within a region can then be committed on a page-by-page basis. It is possible to directly commit a page or a series of pages without first reserving a region of pages, but the page or series of pages committed directly will be aligned on a 64KB boundary. For this reason, it's best to reserve blocks of virtual memory in 64KB chunks, then commit pages within the region as needed.
With the limit of a 32MB virtual memory space per process, this leaves a maximum of 32MB / 64KB – 1 = 511 virtual memory regions that can be reserved before the system will report that it is out of memory. (In this case, the system is actually out of address space, not memory, but the error code is the same.) The following code attempts to allocate 512 1-page blocks of virtual memory:

 // Assume we're on a 1KB page machine 
 #define PAGESIZE 1024   
 for (i = 0; i < 512; i++) 
     pMem[i] = VirtualAlloc (NULL, PAGESIZE, 
 MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE); 
Even if there is 512KB of RAM available in the system, VirtualAlloc will fail before the loop completes. It will run out of virtual address space for the application because each 1KB block will be allocated on a 64KB boundary. Since the code, stack, and local heap for an application must also be mapped into the same 32MB virtual address space, available virtual allocation regions usually top out at around 490.
There are clearly better ways to allocate 1KB blocks than making individual calls to VirtualAlloc. However, this example demonstrates a major difference with Windows CE. In Windows NT, applications have a full 2GB virtual address space to work in. With Windows CE, however, you should be aware of the relatively small 32MB virtual address per application.

Heaps

Clearly, allocating memory on a page basis is inefficient for most applications. To optimize memory use, an application needs to be able to allocate and free memory on a per byte, or at least a per 4 byte basis. The system provides this ability through heaps. Using heaps also isolates an application from having to deal with the differing page sizes of the microprocessors that support Windows CE. An application can simply allocate a block in a heap and the system will deal with the number of pages necessary for the allocation.
Unlike Windows NT or Windows 98, Windows CE only supports allocating fixed blocks in a heap. This simplifies the handling of blocks in the heap but it can lead to the heaps becoming fragmented over time as blocks are allocated and freed. The result can be a heap that is almost empty but still requires a large number of virtual pages since the system can not reclaim a page from the heap unless it is completely free.
As in Windows NT, Windows CE heaps come in two flavors. Each application has a default, or local, heap created by the system when the application is launched. Blocks of memory in the local heap can be allocated, freed, and resized by using the LocalAlloc, LocalFree, and LocalRealloc functions. An application can also create any number of separate heaps. These heaps have the same properties as the local heap but are managed through a separate set of HeapXxx functions.
When an application is loaded by Windows CE, the system always creates a local heap for the application. By default, Windows CE initially reserves 384 pages (on a 1KB page size machine) for the local heap, but only commits the pages as they are allocated. If the application allocates more than the 384KB in the local heap, the system will allocate more space for the local heap. Growing the heap may require a separate, disjointed address space reserved for the additional space on the heap. Applications should not assume that the local heap is contained in one block of virtual address space. Since Windows CE heaps support only fixed blocks, Windows CE implements just the subset of the Win32 local heap functions necessary to allocate, resize, and free fixed blocks on the local heap.
To avoid fragmenting the local heap, it is sometimes better to create a separate heap if you need a series of blocks of memory that will be used for a set amount of time. An example of this would be a text editor that may manage a file by creating a separate heap for each file it is editing. As files are opened and closed, the heaps would be created and destroyed.
Heaps under Windows CE have the same API as those under Windows NT or Windows 98. The only noticeable difference is a lack of support for the HEAP_GENERATE_ EXCEPTIONS flag. Under Windows NT, this flag causes the system to generate an exception if an allocation request cannot be accommodated.
A subtle, but more important difference to the programmer is how Windows CE manages heaps. While its heap API looks like the standard Win32 heap API, Windows CE does not implement the functions as you might expect. For example, the HeapCreate function has parameters that allow a program to specify how much memory to allocate and reserve for a heap. Windows CE ignores these values. In fact, creating a heap does not allocate or reserve any memory. Memory is only reserved and committed when the first block of the heap is allocated.
Under most conditions, the previous comment would be simple nitpicking. However, in low-memory situations, you cannot count on any space in a heap that has not already been allocated by a HeapAlloc despite what you might expect from HeapCreate. The flip side of the way Windows CE manages heaps is that Windows CE does not use the reserved parameter in the HeapCreate call as a hardcoded limit on the size of the heap. Windows CE will accommodate almost any heap allocation request if the memory is available.

The Stack

Windows CE manages a separate stack for every thread in the system. Each individual stack is limited to less than 58KB. Separate threads within one process can grow their stacks up to the 58KB limit. This limit has to do with how Windows CE manages the stack. When a thread is created, Windows CE reserves a 60KB region for the thread's stack. It then commits virtual pages from the top down as the stack grows. As the stack shrinks, the system will, under low-memory conditions, reclaim the unused but still committed pages below the stack. The limit of 58KB comes from the size of the 64KB region dedicated to the stack minus the number of pages necessary to guard the stack against overflow and underflow.
When an application calls a function that needs stack space, Windows CE attempts to commit the pages immediately below the current stack pointer to satisfy the request. If no physical RAM is available, the thread needing the stack space is suspended until the request can be granted. If an application is requested to close because of a low memory condition and it attempts to allocate space on the stack that can't be granted, the system will suspend the thread and the application won't be able to close. The user will be notified of the nonresponding application and will be given the option to purge it from memory.

Static Data

A handy place to squeeze the last useful bytes out of an application is the static data areas of an application. Windows CE allocates two blocks of RAM for the static data of an application, one for the read-write data and one for the read-only data. Since these areas are allocated on a per page basis, there is typically some space left over from the static data up to the next page boundary. A finely tuned Windows CE-based application would make sure there is little or no extra space left over. Sometimes it is better to move a buffer or two into the static data area instead of allocating those buffers dynamically if there is space in the static data area.
Another consideration is that if the application will be ROM-based, as much data as possible should be moved to the read-only static data area. For ROM-based applications, Windows CE does not allocate RAM for the read-only area. Instead, the ROM pages are mapped directly into the virtual address space. This essentially provides unlimited read-only space with no impact on the RAM requirements of the application.
The best place to determine the size of the static data area is to look in the map file that is optionally generated by the linker. The map file is chiefly used to determine the locations of functions and data for debugging purposes, but it also shows the size of the static data, if you know where to look. Figure 4 shows a portion of an example map file generated by Visual C++
®.
This map file indicates the EXE has five sections. Section 0001 is the text segment containing the executable code of the program. Section 0002 contains the read-only static data. Section 0003 contains the read-write static data. Section 0004 contains the fix-up table to support calls to other DLLs. Finally, section 0005 is the resource section containing the application's resources, such as menu and dialog box templates.
The lines to examine are the ones named .data, .bss, and .rdata. The .data section contains the initialized read-write data. If you initialized a global variable as in

 static HINST g_hLoadlib = NULL;
the g_hLoadlib variable would end up in the .data segment. The .bss segment contains the uninitialized read-write data. A buffer defined as

 static BYTE g_ucItems[256];
would end up in the .bss segment. The final .rdata segment contains the read-only data. Static data defined with the const keyword ends up in the .rdata segment. An example of this would be the structures I use for my message lookup tables:

 // Message dispatch table for MainWindowProc
 const struct decodeUINT MainMessages[] = {
     WM_CREATE, DoCreateMain,
     WM_SIZE, DoSizeMain,
     WM_COMMAND, DoCommandMain,
     WM_DESTROY, DoDestroyMain,
 };
The .data and .bss blocks are folded into section 0003, which has a total size of 0x2274 (or 8820) bytes. Rounded up to the next page size, the read-write section ends up taking nine pages with 396 bytes not used. So in this example, placing a buffer or two in the static data section of the application would be essentially free. The read-only segment (section 0002) including .rdata ends up being 0x0842 (or 2114) bytes, which takes up three pages with 958 bytes left over—almost an entire page. In this case, moving 75 bytes of constant data from the read-only segment to the read-write segment will save a page of RAM when the application is loaded.

Memory-Mapped Files

Windows CE 2.0 support of memory-mapped files is somewhat eccentric. Memory-mapped files that are backed up by named files currently are supported only for read-only access. Unnamed memory-mapped files—those typically backed up by the page file—are supported. This is a little strange since Windows CE does not support a page file. In any case, these unnamed files provide a method both for interprocess communication and as a way to allocate virtual memory regions larger than the 32MB slot size limit.
A file can be read as a memory-mapped file by first calling CreateFileForMapping:

 HANDLE CreateFileForMapping(LPCTSTR lpFileName, 
             DWORD dwDesiredAccess,
             DWORD dwShareMode, 
             LPSECURITY_ATTRIBUTES lpSecurityAttributes,
             DWORD dwCreationDisposition, 
             DWORD dwFlagsAndAttributes, 
             HANDLE hTemplateFile);
Windows CE-based applications must use this function when opening files for memory-mapped use instead of using CreateFile as under Windows NT or Windows 98. The parameters for this function are similar to those for CreateFile. The file name is the name of the file to read. The dwDesiredAccess parameter, specifying the access rights to the file, must either be zero or GENERIC_READ. Calling CreateFileForMapping and requesting read-write access with the GENERIC_WRITE flag will result in the function failing with an incorrect parameter error code. The security attributes must be NULL, while the hTemplateFile parameter is ignored by Windows CE.
The handle returned by CreateFileForMapping can then be passed to CreateFileMapping to create a memory-mapped file object, albeit one that is read-only. Finally, a view is created by calling MapViewOfFile, which returns a pointer to the memory-mapped file. The code in Figure 5 demonstrates opening a memory-mapped file.
Using memory-mapped files for interprocess communication isn't helpful if the applications are restricted to read-only access. Windows CE supports unnamed read-write memory-mapped files. Unnamed, or "page file backed," memory-mapped files are not backed by a page file under Windows CE, nor do they have to be unnamed. The name of this type of memory-mapped file comes from Windows NT where an application could use the paging file of the OS to create a huge, sparse array of virtual pages. The memory-mapped file was unnamed since it was not backed up by a real file on the disk. However, the created object can have a name, and that name can be passed to other processes so they can access the same object.
Creating such a memory-mapped file is accomplished by eliminating the call to CreateFileForMapping and passing a –1 in the handle field of CreateFileMapping. Since no file is specified, you must specify the size of the memory-mapped region in the maximum size fields of CreateFileMapping. The following routine creates a 16MB region using a memory-mapped file:

 // Create a 16MB memory-mapped object
 g_hNFileMap = CreateFileMapping((HANDLE)-1, NULL, 
                                  PAGE_READWRITE, 0,
                                  0x1000000, 
                                 TEXT ("Bob"));
 if (g_hNFileMap) 
     // Map in the object
     pNFileMem = MapViewOfFile 
        (g_hNFileMap[g_nNMMCnt], 
        FILE_MAP_WRITE, 0, 0, 0);
The created object is named by passing a Unicode string to CreateFileMapping. In the previous example, the region is named Bob. This name is global so that if another process opens a mapping object with the same name, the two processes will share the same virtual address space. The memory object created by the code shown above doesn't actually commit 16MB of RAM. Instead, only the address space is reserved. Pages are autocommitted as they are accessed. This allows an application to create a huge, sparse array of pages that take up only as much physical RAM as is needed to hold the data.
Figure 6 Unnamed Memory-Mapped Objects
Figure 6 Unnamed Memory-Mapped Objects


The difference between named and unnamed file mapping objects can be seen in Figures 6 and 7. In Figure 6, three unnamed memory-mapped objects have been created and their addresses are shown. In Figure 7, three named memory-mapped objects are created all with the same name. With the unnamed objects, the addresses are all different. In the case of the named objects, all three calls return the same address since they all point to the same named object. Each of the unnamed objects can be deleted by calling UnMapViewOfFile and CloseHandle. The single named memory mapped object will not be deleted until three calls are made to UnMapViewOfFile and CloseHandle, which decrements the use count of the named object to zero.

Figure 7 Named Memory-Mapped Objects
Figure 7 Named Memory-Mapped Objects


Using unnamed memory-mapped files seems perfect for breaking the 32MB virtual memory limit of a Windows CE-based application. Unfortunately, I have found that attempting to create an unnamed file of 32MB or over works, but Windows CE doesn't seem to clean up correctly. Subsequent calls to allocate the same size space work, but the base address is set above the original base returned in the first instance. Repeated calls eventually result in Windows CE failing the call to create the object with an out of memory error code. This seems to be a bug in Windows CE 2.0. Fortunately, creating unnamed memory-mapped files under 32MB seems to work nicely. There also does not seem to be a problem for one process to create multiple memory-mapped objects as long as they stay under the 32MB limit and there is room in the address space for all the objects.
When using memory-mapped files for interprocess communication, processes should pass the name of the region to the second process, not to a pointer. While it is possible for the first process to simply pass a pointer to the mapping region to the other process, this isn't advisable. If the first process frees the memory-mapped file region while the second process is still accessing the file, a general protection fault will occur. Instead, the second process should create a memory-mapped object with the same name as the initial process. The name of the region is passed in CreateFileMapping. If another process opens a memory-mapped file with the same named object, Windows knows to pass a pointer to the same region that was opened by the first process. The system also increments a use count to track the number of opens. This assures a process that the object will remain at least until it closes the object itself.

Selecting the Proper Memory Type

Now that I've looked at the different types of memory, it is time to consider the best use of each. For large blocks of memory, allocating virtual memory directly is best. An application can reserve address space up to the 32MB limit of the application but can only commit the pages necessary at any one time. This approach is flexible, but it leaves you with the burden of page granularity as well as keeping track of the reserved versus committed pages.
The local heap is always handy. It does not have to be created and will grow as necessary to satisfy a request. Fragmentation is the issue here. Consider that applications on a Handheld PC may run for weeks or even months at a time; there is no off on a Handheld PC or a Palm-size PC (P/PC), just suspend. So when you're considering fragmentation, don't assume that a user will open the application, change one item, then close it. A user might just start an application and keep it running so that it pops up quickly.
The advantage of separate heaps is the ability to destroy them when their time is up, nipping the fragmentation problem in the bud. Separate heaps also handle block resizing better since they can relocate blocks to find room to grow the block. A minor problem with separate heaps is the need to create and destroy them manually. Another thing to remember about separate heaps is that Windows CE does not reserve virtual address space when a heap is created. This can become an issue if your application uses a large amount of the virtual space available to the app.
The static data area is a great place to slip in a buffer or two for free since the page is going to be allocated anyway. The key to managing the static data is to make the size of the static data segments close to but not over the page size of your target processor. For applications written for the Handheld PC or P/PC, consider the 1,024 byte page size of the NEC MIPS 4100 and Hitachi SH3 processors as the default. Sometimes it is better to move constant data from the read-only segment to the read-write segment if it saves a page in the read-only segment. The only time you would not do this is if the application will be burned into ROM. In that case, more constant data is better since it does not take up RAM.
The stack is simple to use and always around. The only considerations are the maximum size of the stack and the problems of growing the stack in a low memory condition. Make sure your application does not require large amounts of stack space to shut down. If the system suspends a thread in your application while it is being shut down, the user will probably lose data. That won't help the customer satisfaction numbers.
Memory-mapped objects are great for allocating a huge, sparse array of virtual memory pages. As long as an application simply needs the large address space and not that much committed memory, memory-mapped files are handy. They are also handy for interprocess communication since multiple processes can share the same memory-mapped object.

Managing Low Memory Conditions

Even for applications that have been fine-tuned to minimize their memory use, there are going to be times when the system runs very low on RAM. Windows CE-based applications operate in an almost perpetual low memory environment. Because of this, Windows CE provides a number of methods to manage scarce memory among the number of applications running in the system.
The first and most obvious addition to Windows CE is the WM_HIBERNATE message. Windows CE sends this message to all top-level windows that have a button on the task bar. Task bar buttons are assigned to windows that have the overlapped style, or have neither the WS_POPUP nor the WS_CHILD style, or have the WS_VISIBLE style. These qualifications should allow most applications to have at least one window that receives a WM_HIBERNATE message. An exception would be an application that does not really terminate, but simply hides its windows. This allows an application to start quickly since it only has to show its windows, but the application will occupy RAM even when the user thinks it is closed. This is exactly the type of application design that should not be used under Windows CE. If you choose this approach, your app must act like it's always in hibernate mode when hidden since it will never receive a WM_HIBERNATE message.
Windows CE sends WM_HIBERNATE messages to the top-level windows in reverse Z-order until enough memory is freed to get the available memory above a preset threshold. When an app receives a WM_HIBERNATE message, it should reduce its memory footprint to as small as possible while retaining its internal state. This can involve releasing cached data, freeing GDI objects such as fonts, bitmaps, or brushes, and destroying window controls.
If sending WM_HIBERNATE messages to background applications doesn't free enough memory to move the system out of a limited memory state, a WM_HIBERNATE message will be sent to the application in the foreground. If part of your hibernation routine is to destroy controls on your window, you should make sure that you are not the foreground application. Disappearing controls do not give the user a warm fuzzy feeling.

Memory Thresholds

Windows CE monitors the free RAM in the system and responds differently as less and less RAM is available. As the available RAM is reduced, Windows CE first sends WM_HIBERNATE messages, then it starts to limit the size of allocations possible. Figure 8 shows the free memory levels that Windows CE uses to trigger low memory events in the system. Windows CE defines four memory states: normal, limited, low, and critical. The memory state of the system depends on how much free memory is available to the system as a whole. These limits are higher for 4KB-page systems because those systems have less granularity in allocations. The effect of these memory states is to, in effect, share the remaining wealth.
First, WM_HIBERNATE messages are sent to the applications to ask them to reduce their memory footprint. If an allocation will result in the system entering the low memory state, the system will display the Low Memory dialog. This dialog asks the user which application should be closed or if memory should be taken from the object store. If the user chooses to close an application, the top-level window of the application is sent a WM_CLOSE message. If an application does not close within a predetermined amount of time (around eight seconds), the system will terminate the threads of the application and remove it from memory. If closing the application or taking RAM from the object store still does not satisfy the memory request, the low memory dialog is displayed again and the process is repeated.
In the low and critical memory states, the amount of memory an application can allocate is limited. In these states, a request for virtual memory will be refused even if there is memory available to satisfy the request if the request is larger that what is allowed. Remember that not just virtual memory allocations are limited; allocations on the heap and stack will be rejected if those allocations require virtual memory allocations above the allowable limits.
It should go without saying that applications need to check the return codes of any memory allocation call. Check the return codes from calls that allocate memory. There is a far greater chance of a memory allocation call failing under Windows CE than under Windows NT or Windows 98. Applications must be written to react gracefully to rejected memory allocations.
The source for a sample application called MemDemo is available from the code link at the top of this page. It's a small, Windows CE-based application that demonstrates the topics discussed here. The application can dump the page state of any region of memory as well as the global memory status. It will create named and unnamed page file backed memory-mapped regions as well as opening a file as a read-only memory-mapped file.
While not every Win32 memory management API is supported by Windows CE, there is clearly more than enough support to use the limited memory of a Windows CE device to the fullest. When programming a Windows CE-based application, treat it as you would any Win32-based application, but be aware of the subtleties of the Windows CE platform.

From the May 1998 issue of Microsoft Systems Journal. Get it at your local newsstand, or better yet, subscribe.


For related information see: Microsoft Windows CE Memory Use, http://premium.microsoft.com/msdn/library/bkgrnd/html/memdrft2.htm.
Also check http://www.microsoft.com/msdn for daily updates on developer programs, resources and events.

© 1998 Microsoft Corporation. All rights reserved.
Terms of Use
.

© 2014 Microsoft Corporation. All rights reserved. Contact Us |Terms of Use |Trademarks |Privacy Statement
Microsoft