Conventional memory

From Wikipedia, the free encyclopedia

Conventional memory is the first 640 kilobytes of the memory on IBM PC compatible systems.

1 640 kB barrier
2 Additional memory
3 Memory optimisation
4 DOS extenders
5 Modern Memory Limitations
- 5.1 The 32-bit 3.2 gigabyte barrier
6 See also
7 References

[edit] 640 kB barrier

The 640 kB barrier is an architectural limitation of IBM and IBM Compatible PCs. The Intel 8088 CPU, used in the original IBM PC, was able to address 1024 kB (1 MB or 2²⁰ bytes), as the chip offered 20 address lines. The lower limit was due to hardware mapping:

In order for a CPU to be able to interact with both the RAM and other hardware, some portion of the address space must be set aside for hardware use only, such as for accessing a video card's frame buffer or to run software stored in an expansion card's ROM. As originally conceived, the memory below 640 kB was for RAM, and the 384 kB above reserved for hardware use. Because the original designers could not know what future applications might arise for the hardware, much of the 384k was unused but reserved in the event that it was needed by hardware. A similar reserved hardware space exists in other system designs as well; for the original Apple II computer, the CPU could address a total of 64 kB, with 48 kB intended for RAM and the upper 16 kB reserved for hardware use.

At the time of the PC's release in 1981, 640 kB would have seemed more than plenty for the typical user.^{[citation needed]} The first IBM PC motherboards were available with only 64 kB to 256 kB installed, and it took a few years until most new PCs had even that much memory installed through memory expansion cards. The most popular existing microcomputer when the PC appeared, the Apple II+, had only 64 kB in the most common configuration and could not be easily expanded beyond this; the value of compatibility of succeeding generations of microcomputers was not yet perceived as clearly back then as it was in the mid-1980s, so that probably few people at IBM or elsewhere expected the IBM PC architecture to last more than a few years in the market.^{[citation needed]}

The design of the original IBM PC placed the Color Graphics Adapter memory map at the location of 640 kB, and other hardware at higher locations (see Upper Memory Area for more details). The need for more RAM grew faster than the needs of hardware to utilize the reserved addresses, which resulted in RAM eventually being mapped into these unused upper areas to utilize all available addressible space. This introduced a reserved "hole" (or several holes) into the set of addresses occupied by hardware that could be used for arbitrary data, avoiding such a hole was difficult and ugly and not supported by MS-DOS or most programs that MS-DOS could run.

To maintain compatibility with older OSes, the 640 kB barrier remained part of the PC design even after the 8088 had been replaced with the Intel 286 processor, which could address up to 16 MB of memory. The 1MB barrier also remained as long as the 286 was running in compatibility mode, as MS-DOS forced assumptions about how the segment and offset registers overlapped such that addresses with more than 20 bits were unsupported. It is still present in IBM PC compatibles today if they are running MS-DOS, and even in the most modern Windows-based PCs the RAM still has a "hole" in the area between 640 and 1024 kilobytes, which however is invisible to application programs thanks to paging and virtual memory.^{[citation needed]}

[edit] Additional memory

One technique used on early IBM XT computers was to ignore the Color Graphics Adapter (CGA) memory map and push the limit up to the start of the Monochrome Display Adapter (MDA). Sometimes software or a custom address decoder was used so that attempts to use the CGA went to the MDA memory. This moved the barrier to 704 kB[1].

Memory managers on 386-based systems (such as QEMM or MemoryMax in DR-DOS) could achieve the same effect, adding conventional memory at 640 kB and moving the barrier to 704 kB or 736 kB. Only CGA could be used in this situation, because Enhanced Graphics Adapter (EGA) used this memory for the bit-mapped graphics buffer.

The AllCard, an add-on memory management unit (MMU) for XT-class computers, allowed normal memory to be mapped into the A0000-EFFFF (hex) address range, giving up to 952 kB for DOS programs. Programs such as Lotus 1-2-3, which accessed video memory directly, needed to be patched to handle this memory layout. Therefore, the 640 kB barrier was removed at the cost of hardware compatibility.

It was also possible to use DOS's utility for console redirection, setty, to direct output to a dumb terminal or another computer running a terminal emulator. The video card could then be removed completely, and assuming the BIOS still permitted the machine to boot, the system could achieve a total memory of 960k of ram. This also required that the system have at least 2MB of physical memory in the machine. This procedure was tested on a 486 with IBM PC DOS 7.0. The total operating system footprint was around 20k, most of DOS residing in the HMA.

[edit] Memory optimisation

As DOS applications grew larger and more complex in the late 1980s, it became common practice to free up conventional memory, by moving device drivers and Terminate and Stay Resident programs into Upper Memory Blocks in the Upper Memory Area at boot, in order to maximize the conventional memory available for applications. This had the advantage of not requiring hardware changes, and preserved application compatibility. This feature began with DR DOS 5 and was later implemented in MS DOS 5.

Most users used the free emm386 driver provided in DOS 5, but third party products from companies such as QEMM, also proved popular. The process of optimisation was eccentric, in the sense that because the sizes of the UMBs would vary, getting best results required moving equivalent sized drivers, into equivalent sized blocks. To this end, manual tuning of the process typically produced best results, in the sense of greatest free conventional memory.

In DOS 6, Microsoft introduced memmaker, which automated this process of block matching, matching the functionality third party memory managers offered.

[edit] DOS extenders

The barrier was only overcome with the arrival of DOS extenders, which allowed DOS applications to run in extended memory, but these were not very widely used outside the computer game area. The first PC operating systems to integrate such technology were Compaq DOS 3.31 (via CEMM) and Windows/386 2.1, both released in 1988. Since the 80286 version of Windows 2.0 (Windows/286), Windows applications did not suffer from the 640 kB barrier.

Prior to DOS extenders, if a user installed additional memory and wished to use it under DOS, they would first have to install and configure drivers to support either expanded memory specification (EMS) or extended memory specification (XMS).

EMS was a specification available on all PCs, including 8086's or 8088's which allowed add-on hardware to page small chunks of memory in and out of the "real mode" addressing space. (0x0400-0xFFFF). This required that a hole in real memory be available, typically (0xE000-0xEFFF). A program would then have to explicitly request the page to be accessed before using it. These memory locations could then be used arbitrarily until replaced by another page. This is very similar to modern virtual memory except that the operating system handles all page operations and the programmer, for the most part, does not have to consider this.

XMS provided a crude protocol which allowed the client program to load a custom protected mode kernel. This was available on the 286 and newer processors. The problem with this approach is that while in protected mode, DOS calls could not be made. The work around was to implement a callback mechanism. On the 286, this was a major problem. The 386, which introduced "Virtual86 mode", allowed the guest kernel to emulate the 8086 and run the host operating system without having to actually force the processor back into "real mode".

The latest DOS extension was DPMI, a more advanced version of XMS which provided many of the services of a modern kernel, obviating the need to write a custom kernel. It also permitted multiple protected mode clients. This was the standard target environment for the DOS port of the GCC compilers.

There were a number of other common dos extenders. The most notable of which was the runtime environment for the Watcom compilers, Dos4GW, which was very common in games. Such a game would consist of either a DOS4GW 32-bit kernel, or a stub which loaded a DOS4GW kernel located in the same directory and a 32-bit "linear executable". Utilities are available which can strip DOS4GW out of such a program and allow the user to experiment with any of the several, and perhaps improved, DOS4GW clones.

[edit] Modern Memory Limitations

With modern operating systems and 32-bit machines the distinction between conventional and extended memory is rarely relevant for anybody except people who write boot loaders and operating systems, since all memory can be accessed uniformly in protected mode. Most users are therefore unaware of the special significance the first 640 kB of PC memory has in the history of the development of personal computers.

However, the problems of memory reserved for hardware use have not disappeared. Instead as the addressible memory space increased, the reserved section has been pushed ever higher into what is assumed an addressible space that most computer users will never need. As PCs have moved from having kilobytes to megabytes to gigabytes of RAM installed, a new 32-bit barrier is becoming familiar to high-performance hardware users.

[edit] The 32-bit 3.2 gigabyte barrier

With 32-bit hardware, starting at approximately 3.2 gigabytes up to 4.0 gigabytes is a region of memory reserved for hardware use only. This barrier only becomes evident when more than 3 gigabytes of RAM have been installed in a system. Installation of an additional gigabyte leads to only 256 megabytes of it being accessible.

This barrier is flexible and there can be less RAM available than 3.2 gigabytes, due to hardware being able to allocate resizable memory apertures that are reserved by the hardware and exclude addressible RAM. When a large amount of video RAM is installed such as by using two 512 megabyte nVidia graphics cards in SLI mode, the available RAM can be reduced to as low as 2.6 gigabytes.

The only way to surmount this limitation is to increase the physical address space. PAE, introduced in the Pentium Pro, extended the physical address space to 36-bit, though this must also be coupled with a PAE-enabled (or 64-bit) operating system for the capabilities of the hardware to become available. The 64-bit address space still has its own reserved hardware space, but with 17,179,869,184 gigabytes of addressible memory it is going to be a very long time before that limit is approached again.