NUMAlink

From Wikipedia, the free encyclopedia

NUMAlink is a high-speed low-latency switched fabric computer bus used as a shared memory computer cluster processor interconnection in Silicon Graphics computer systems. NUMAlink was developed by SGI for their Origin and Onyx systems. It was initially branded as "CrayLink" during SGIs brief acquisition of Cray Research.

For computer clusters, low latency of the interconnect is often more important to overall performance than overall bandwidth. This is more an issue for applications that pass small messages. For instance, gigabit ethernet has performance on the order of 100 MB/s, but typical latencies of 30 usecs even for one-word messages. This is due to the overhead of the Ethernet protocol stack, which has to encapsulate the message in a standardized package and then unpackage it at the far end.

NUMAlink, like other products aimed as the same market space, attempt to improve performance by dramatically reducing the packet overhead. Typically this is accomplished by using a much smaller minimum packet size, and using circuit switched networks that do not have to be actively routed during transport (the route is set up only once). SGI claims particularly impressive numbers for NUMAlink, stating that the typical short-message overhead is only 1 usec, half that of competing systems.

Latency directly effects the "efficiency" of a system, one of the important measures used in Linpack for benchmarking supercomputer installations. NUMAlink offered an average of 84% efficiency on the TOP500 list ^[1], while QsNet and Infiniband reached 75%, Myrinet 63%, and 59% for gigabit ethernet.

Moreover, NUMAlink is extremely fast. The basic system offers 3.2 GB/s unidirectionally, about twice that of most similar systems, and 32 times that of gigabit ethernet. Fully expanded InfiniBand systems, that is "quad-rate 12X" systems, offer up to 12 GB/s, but it appears no such solution is actually in use.

There are several generation of NUMAlink technology.

NL2 was used on Origin 2000.
NL3 was used on Origin 3000 and Altix 3000.
NL4 is used on Altix 4700.