DOS memory management

This article is about a class of DOS device drivers. For the general concept, see: memory management.

Physical memory areas of the IBM PC family.

In IBM PC compatible computing, DOS memory management refers to software and techniques employed to give applications access to more than 640 kibibytes (640*1024 bytes) (kiB) of "conventional memory". The 640 KiB limit was specific to the IBM PC and close compatibles; other machines running MS-DOS had different limits, for example the Apricot PC could have up to 768 KiB and the Sirius Victor 9000, 896 KiB. Memory management on the IBM family was made complex by the need to maintain backward compatibility to the original PC design^[1] and real-mode DOS, while allowing computer users to take advantage of large amounts of low-cost memory and new generations of processors. Since DOS has given way to Microsoft Windows and other 32-bit operating systems not restricted by the original arbitrary 640 KiB limit of the IBM PC, managing the memory of a personal computer no longer requires the user to manually manipulate internal settings and parameters of the system.

The 640 KiB limit imposed great complexity on hardware and software intended to circumvent it; the physical memory in a machine could be organised as a combination of base or conventional memory, upper memory, high memory (not the same as upper memory), extended memory, and expanded memory, all handled in different ways.

Conventional memory

The Intel 8088 processor used in the original IBM PC had 20 address lines and so could directly address 1 MiB (2²⁰ bytes) of memory. Different areas of this address space were allocated to different kinds of memory used for different purposes. Starting at the lowest end of the address space, the PC had read/write random access memory (RAM) installed, which was used by DOS and application programs. The first part of this memory was installed on the motherboard of the system (in very early machines, 64 KiB, later revised to 256 KiB). Additional memory could be added with cards plugged into the expansion slots; each card contained straps or switches to control what part of the address space access memory and devices on that card.

On the IBM PC, all the address space up to 640 KiB was available for RAM. This part of the address space is called "conventional memory" since it is accessible to all versions of DOS automatically on start up. Normally expansion memory is set to be contiguous in the address space with the memory on the motherboard. If there was an unallocated gap between motherboard memory and the expansion memory, the memory would not be automatically detected as usable by DOS.

Upper memory area

The upper memory area (UMA) refers to the address space between 640 KiB and 1024 KiB (0xA0000–0xFFFFF). Three 128 KiB regions were defined in this area. The 128 KiB region between 0xA0000 and 0xBFFFF was reserved for video adapter screen memory. The physical address space between 0xC0000 and 0xDFFFF was reserved for device BIOS ROMs, and special RAM usually shared with physical devices (for example, shared memory for a network adapter). The IBM PC reserved the uppermost 128 KiB of the address space from 0xE0000 to 0xFFFFF for the ROM BIOS and Cassette BASIC read-only memory (ROM).

For example, the monochrome video adapter memory area ran from 704 to 736 KiB (0xB0000–0xB7FFF). If only a monochrome display adapter was used, the address space between 0xA0000 and 0xAFFFF could be used for RAM, which would be contiguous with the conventional memory.^[2]

The system BIOS ROMs must be at the upper end of the address space because the CPU starting address is fixed by the design of the processor. The starting address is loaded into the program counter of the CPU after a hardware reset and must have a defined value that endures after power is interrupted to the system. On reset or power up, the CPU loads the address from the system ROM and then jumps to a defined ROM location to begin executing the system power-on self-test, and eventually load an operating system.

Since an expansion card such as a video adapter, hard drive controller, or network adapter could use allocations of memory in many of the upper memory areas, configuration of some combinations of cards required careful reading of documentation, or experimentation, to find card settings and memory mappings that worked. Mapping two devices to use the same physical memory addresses could result in a stalled or unstable system.^[2] Not all addresses in the upper memory area were used in a typical system; unused physical addresses would return undefined and system-dependent data if accessed by the processor.

Expanded memory

As memory prices declined, application programs such as spreadsheets and computer-aided drafting were changed to take advantage of more and more physical memory in the system. Virtual memory in the 8088 and 8086 was not supported by the processor hardware, and disk technology of the time would make it too slow and cumbersome to be practical. Expanded memory was a system that allowed application programs to access more RAM than directly visible to the processor's address space. The process was a form of bank switching. When extra RAM was needed, driver software would temporarily make a piece of expanded memory accessible to the processor; when the data in that piece of memory was updated, another part could be swapped into the processor's address space. For the IBM PC and IBM PC/XT, with only 20 address lines, special-purpose expanded memory cards were made containing perhaps a mebibyte, or more, of expanded memory, with logic on the board to make that memory accessible to the processor in defined parts of the 8088 address space.

Allocation and use of expanded memory was not transparent to application programs. The application had to keep track of which bank of expanded memory contained a particular piece of data, and when access to that data was required, the application had to request (through a driver program) the expanded memory board to map that part of memory into the processor's address space. Although applications could use expanded memory with relative freedom, many other software components such as drivers and TSRs were still normally constrained to reside within the 640K "conventional memory" area, which soon became a critically scarce resource.

The 80286 and the high memory area

When the IBM PC/AT was introduced, the segmented memory architecture of the Intel family processors had the byproduct of allowing slightly more than 1 MiB of memory to be addressed in the "real" mode. Since the 80286 had more than 20 address lines, certain combinations of segment and offset could point into memory above the 0x0100000 (2²⁰) location. The 80286 could address up to 16 MiB of system memory, thus removing the behavior of memory addresses "wrapping around". Since the required address line now existed, the combination F800:8000 would no longer point to the physical address 0x0000000 but the correct address 0x00100000.

As a result some DOS programs would no longer work. To maintain compatibility with the PC and XT behavior, the AT included an A20 line gate (Gate A20) that made memory addresses on the AT wrap around to low memory as they would have on an 8088 processor. This gate could be controlled, initially through the keyboard controller, to allow running programs which were designed for this to access an additional 65520 bytes (64 KiB of memory in real mode. At boot time, the BIOS first enables A20 when counting and testing all of the system's memory, and disables it before transferring control to the operating system. Enabling the A20 line is one of the first steps a protected mode x86 operating system does in the bootup process, often before control has been passed onto the kernel from the bootstrap (in the case of Linux, for example).

The high memory area (HMA) is the RAM area consisting of the first 64 KiB, minus 16 bytes, of the extended memory on an IBM PC/AT or compatible microcomputer.

Originally, the logic gate was a gate connected to the Intel 8042 keyboard controller. Controlling it was a relatively slow process. Other methods have since been added to allow for more efficient multitasking of programs which require this wrap-around with programs that access all of the system's memory. There was at first a variety of methods, but eventually the industry settled on the PS/2 method of using a bit in port 92h to control the A20 line.

Disconnecting A20 would not wrap all memory accesses above 1 MiB, just those in the 1 MiB, 3 MiB, 5 MiB, etc. ranges. Real mode software only cared about the area slightly above 1 MiB, so Gate A20 was enough.

Virtual 8086 mode, introduced with the Intel 80386, allows the A20 wrap-around to be simulated by using the virtual memory facilities of the processor: physical memory may be mapped to multiple virtual addresses thus allowing the memory mapped at first mebibyte of virtual memory may be mapped again in the second mebibyte of virtual memory. The operating system may intercept changes to Gate A20 and make corresponding changes to the virtual memory address space, which also makes irrelevant the efficiency of Gate-A20 toggling.

The first user of the HMA among Microsoft products was Windows/286 2.1 in 1988, which introduced the HIMEM.SYS device driver. Starting with versions 5.0 of DR-DOS (1990) and of MS-DOS (1991), parts of the operating system could be loaded into HMA as well, freeing up to 46 KiB of conventional memory. Other components, such as device drivers and TSRs, could be loaded into the upper memory area (UMA).

A20 handler

The A20 handler is software controlling access to the high memory area. Extended memory managers usually provide this functionality. In DOS, high memory area managers, such as HIMEM.SYS had the extra task of managing A20 and provided an API for opening/closing A20. DOS itself could utilize the area for some of its storage needs, thereby freeing up more conventional memory for programs. This functionality was enabled by the "DOS=HIGH" directive in the CONFIG.SYS configuration file.

A20 gate on later processors

The Intel 80486 and Pentium added a special pin named A20M#, which when asserted low forces bit 20 of the physical address to be zero for all on-chip cache or external memory accesses. This was necessary since the 80486 introduced an on-chip cache, and therefore masking this bit in external logic was no longer possible. Software still needs to manipulate the gate and must still deal with external peripherals (the chipset) for that.^[3]

80386 and subsequent processors

Intel processors from the 386 onward allowed a virtual 8086 mode, which simplified the hardware required to implement expanded memory for DOS applications. Expanded memory managers such as Quarterdeck's QEMM product and Microsoft's EMM386 supported the expanded memory standard without requirement for special memory boards.

On 386 and subsequent processors, memory managers like QEMM might move the bulk of the code for a driver or TSR into extended memory and replace it with a small fingerhold that was capable of accessing the extended-memory-resident code. They might analyze memory usage to detect drivers that required more RAM during startup than they did subsequently, and recover and reuse the memory that was no longer needed after startup. They might even remap areas of memory normally used for memory-mapped I/O. Many of these tricks involved assumptions about the functioning of drivers and other components. In effect, memory managers might reverse-engineer and modify other vendors' code on the fly. As might be expected, such tricks did not always work. Therefore, memory managers also incorporated very elaborate systems of configurable options, and provisions for recovery should a selected option render the PC unbootable (a frequent occurrence).

Installing and configuring a memory manager might involve hours of experimentation with options, repeatedly rebooting the machine, and testing the results. But conventional memory was so valuable that PC owners felt that such time was well-spent if the result was to free up 30 KiB or 40 KiB of conventional memory space.

Extended memory

In the context of IBM PC compatible computers, extended memory refers to memory in the address space of the 80286 and subsequent processors, beyond the 1 mebibyte limit imposed by the 20 address lines of the 8088 and 8086. Such memory is not directly available to DOS applications running in the so-called "real mode" of the 80286 and subsequent processors. This memory is only accessible in the protected or virtual modes of 80286 and higher processors.

See also

• Global EMM Import Specification (GEMMIS)
• Virtual DMA Services (VDS)
• Virtual Control Program Interface (VCPI)
• Extended Virtual Control Program Interface (XVCPI)
• DOS Protected Mode Interface (DPMI)
• DOS Protected Mode Services (DPMS)
• Helix Cloaking

References

1. ↑ Scott Mueller, 'Upgrading and Repairing PCs 4th Edition, Que, 1994, ISBN 1-56529-932-9 chapter 7
2. 1 2 Scott Mueller, Upgrading and repairing PC's, 11th Edition, Que Corporation, 1999 ISBN 0-7897-1903-7, Chapter 6 Memory
3. ↑ Tom Shanley,Protected mode software architecture, Taylor & Francis, 1996, ISBN 0-201-55447-X, page 60

External links

• Microsoft support: Overview of Memory-Management Functionality in MS-DOS
• Computer Chronicles (1990). "High Memory Management". From the Internet Archive.