Message Signaled Interrupts

Message Signaled Interrupts (MSI) are an alternative in-band method of signaling an interrupt, using special in-band messages to replace traditional out-of-band assertion of dedicated interrupt lines. While more complex to implement in a device, message signaled interrupts have some significant advantages over pin-based out-of-band interrupt signaling.

Message signaled interrupts are supported in PCI bus since its version 2.2, and in later available PCI Express bus. Some non-PCI architectures also use message signaled interrupts.

Overview

Traditionally, a device has an interrupt line (pin) which it asserts when it wants to signal an interrupt to the host processing environment. This traditional form of interrupt signaling is an out-of-band form of control signaling since it uses a dedicated path to send such control information, separately from the main data path. MSI replaces those dedicated interrupt lines with in-band signaling, by exchanging special messages that indicate interrupts through the main data path. In particular, MSI allows the device to write a small amount of interrupt-describing data to a special memory-mapped I/O address, and the chipset then delivers the corresponding interrupt to a processor.^[1][2][3]

A common misconception with MSI is that it allows the device to send data to a processor as part of the interrupt. The data that is sent as part of the memory write transaction is used by the chipset to determine which interrupt to trigger on which processor; that data is not available for the device to communicate additional information to the interrupt handler.^[1][2][3]

As an example, PCI Express does not have separate interrupt pins at all; instead, it uses special in-band messages to allow pin assertion or deassertion to be emulated. Some non-PCI architectures also use MSI; as another example, HP GSC devices do not have interrupt pins and can generate interrupts only by writing directly to the processor's interrupt register in memory space. The HyperTransport protocol also supports MSI.^[4]

Advantages

While more complex to implement in a device, message signaled interrupts have some significant advantages over pin-based out-of-band interrupt signaling. On the mechanical side, fewer pins makes for a simpler, cheaper, and more reliable connector. While this is no advantage to the standard PCI connector, PCI Express takes advantage of these savings.

MSI increases the number of interrupts that are possible. While conventional PCI was limited to four interrupts per card (and, because they were shared among all cards, most are using only one), message signaled interrupts allow dozens of interrupts per card, when that is useful.^[1]

There is also a slight performance advantage. In software, a pin-based interrupt could race with a posted write to memory. That is, the PCI device would write data to memory and then send an interrupt to indicate the DMA write was complete. However, a PCI bridge or memory controller might buffer the write in order to not interfere with some other memory use. The interrupt could arrive before the DMA write was complete, and the processor could read stale data from memory. To prevent this race, interrupt handlers were required to read from the device to ensure that the DMA write had finished. This read had a moderate performance penalty. An MSI write cannot pass a DMA write, so the race is eliminated.^[5]

MSI types

PCI defines two optional extensions to support Message Signaled Interrupts, MSI and MSI-X. While PCI Express is compatible with legacy interrupts on the software level, it requires MSI or MSI-X.

MSI

MSI (first defined in PCI 2.2) permits a device to allocate 1, 2, 4, 8, 16 or 32 interrupts. The device is programmed with an address to write to (generally a control register in an interrupt controller), and a 16-bit data word to identify it. The interrupt number is added to the data word to identify the interrupt.^[1] Some platforms such as Windows do not use all 32 interrupts but only use up to 16 interrupts.^[6]

MSI-X

MSI-X (first defined in PCI 3.0) permits a device to allocate up to 2048 interrupts. The single address used by original MSI was found to be restrictive for some architectures. In particular, it made it difficult to target individual interrupts to different processors, which is helpful in some high-speed networking applications. MSI-X allows a larger number of interrupts and gives each one a separate target address and data word. Devices with MSI-X do not necessarily support 2048 interrupts but at least 64 which is double the maximum MSI interrupts.^[3][7][8][9]

Optional features in MSI (64-bit addressing and interrupt masking) are also mandatory with MSI-X.

x86 systems

On Intel systems, the LAPIC must be enabled for the PCI (and PCI Express) MSI/MSI-X to work, even on uniprocessor (single core) systems.^[10][11] In these systems, MSIs are handled by writing the interrupt vector directly into the LAPIC of the processor/core that needs to service the interrupt. The Intel LAPICs of 2009 supported up to 224 of MSI-based interrupts.^[11] According to a 2009 Intel benchmark using Linux, using MSI reduced the latency of interrupts by a factor of almost three when compared to I/O APIC delivery.^[12]

Some motherboard implementations, particularly among those with PCI to PCI-X bridges, have non-functional MSI support. The AMD-8131 and some ServerWorks PCI Express chipsets were among those with problems, at least under some versions of Linux.^[13] It has also been observed that some motherboards with Nvidia nForce chipsets have broken MSI support. As a result, when NVidia enabled the MSI support on their graphics card drivers for the first time, many users of the nForce-based motherboards complained of instability, what was eventually traced back to the MSI implementation of the motherboard. Users complained of errors ranging from relatively harmless unresponsive GPUs in Windows Vista and 7 that eventually recover, USB devices intermittently ceasing to work in Linux when the GPU is tasked with heavy workloads, to hard freezes, blue screens and kernel panics on both platforms. Later versions of drivers blacklist these platforms and will not activate MSI on them.

Operating system support

In the Microsoft family of operating systems, Windows Vista and later versions have support for both MSI and MSI-X. Support was added in the Longhorn development cycle around 2004.^[14] MSI is not supported in earlier versions like Windows XP or Windows Server 2003.^[15]

Solaris Express release 6/05 added support for MSI an MSI-X as part of their new DDI Interrupt framework.^[16]

FreeBSD 6.3 and 7.0 added support for MSI and MSI-X.^[17]

Linux gained support for MSI and MSI-X around 2003.^[18] Linux kernel versions before 2.6.20 are known to have serious bugs and limitations in their implementation of MSI/MSI-X.^[19]

References

External links

• Introduction to Message-Signaled Interrupts - MSDN
• Linux MSI HOWTO