Blue Gene

A Blue Gene/P supercomputer at Argonne National Laboratory
Hierarchy of Blue Gene processing units

Blue Gene is an IBM project aimed at designing supercomputers that can reach operating speeds in the PFLOPS (petaFLOPS) range, with low power consumption.

The project created three generations of supercomputers, Blue Gene/L, Blue Gene/P, and Blue Gene/Q. Blue Gene systems have often led the TOP500[1] and Green500[2] rankings of the most powerful and most power efficient supercomputers, respectively. Blue Gene systems have also consistently scored top positions in the Graph500 list.[3] The project was awarded the 2009 National Medal of Technology and Innovation.[4]

As of 2015, IBM seems to have ended the development of the Blue Gene family[5] though no public announcement has been made. IBM's continuing efforts of the supercomputer scene seems to be concentrated around OpenPower, using accelerators such as FPGAs and GPUs to battle the end of Moore's law.[6]

History

In December 1999, IBM announced a US$100 million research initiative for a five-year effort to build a massively parallel computer, to be applied to the study of biomolecular phenomena such as protein folding.[7] The project had two main goals: to advance our understanding of the mechanisms behind protein folding via large-scale simulation, and to explore novel ideas in massively parallel machine architecture and software. Major areas of investigation included: how to use this novel platform to effectively meet its scientific goals, how to make such massively parallel machines more usable, and how to achieve performance targets at a reasonable cost, through novel machine architectures. The initial design for Blue Gene was based on an early version of the Cyclops64 architecture, designed by Monty Denneau. The initial research and development work was pursued at IBM T.J. Watson Research Center and led by William R. Pulleyblank.[8]

At IBM, Alan Gara started working on an extension of the QCDOC architecture into a more general-purpose supercomputer: The 4D nearest-neighbor interconnection network was replaced by a network supporting routing of messages from any node to any other; and a parallel I/O subsystem was added. DOE started funding the development of this system and it became known as Blue Gene/L (L for Light); development of the original Blue Gene system continued under the name Blue Gene/C (C for Cyclops) and, later, Cyclops64.

In November 2004 a 16-rack system, with each rack holding 1,024 compute nodes, achieved first place in the TOP500 list, with a Linpack performance of 70.72 TFLOPS.[1] It thereby overtook NEC's Earth Simulator, which had held the title of the fastest computer in the world since 2002. From 2004 through 2007 the Blue Gene/L installation at LLNL[9] gradually expanded to 104 racks, achieving 478 TFLOPS Linpack and 596 TFLOPS peak. The LLNL BlueGene/L installation held the first position in the TOP500 list for 3.5 years, until in June 2008 it was overtaken by IBM's Cell-based Roadrunner system at Los Alamos National Laboratory, which was the first system to surpass the 1 PetaFLOPS mark. The system was built in Rochester, MN IBM plant.

While the LLNL installation was the largest Blue Gene/L installation, many smaller installations followed. In November 2006, there were 27 computers on the TOP500 list using the Blue Gene/L architecture. All these computers were listed as having an architecture of eServer Blue Gene Solution. For example, three racks of Blue Gene/L were housed at the San Diego Supercomputer Center.

While the TOP500 measures performance on a single benchmark application, Linpack, Blue Gene/L also set records for performance on a wider set of applications. Blue Gene/L was the first supercomputer ever to run over 100 TFLOPS sustained on a real world application, namely a three-dimensional molecular dynamics code (ddcMD), simulating solidification (nucleation and growth processes) of molten metal under high pressure and temperature conditions. This achievement won the 2005 Gordon Bell Prize.

In June 2006, NNSA and IBM announced that Blue Gene/L achieved 207.3 TFLOPS on a quantum chemical application (Qbox).[10] At Supercomputing 2006,[11] Blue Gene/L was awarded the winning prize in all HPC Challenge Classes of awards.[12] In 2007, a team from the IBM Almaden Research Center and the University of Nevada ran an artificial neural network almost half as complex as the brain of a mouse for the equivalent of a second (the network was run at 1/10 of normal speed for 10 seconds).[13]

Name

The name Blue Gene comes from that it was originally designed to do, help biologists understand the processes of protein folding and gene development.[14] "Blue" is a traditional moniker that IBM uses for many of its products and the company itself. The original Blue Gene design was renamed "Blue Gene/C" and eventually Cyclops64. The "L" in Blue Gene/L comes from "Light" as that design's original name was "Blue Light". The "P" version was designed to be a petascale design. "Q" is just the letter after "P". There is no Blue Gene/R.[15]

Major features

The Blue Gene/L supercomputer was unique in the following aspects:[16]

Architecture

The Blue Gene/L architecture was an evolution of the QCDSP and QCDOC architectures. Each Blue Gene/L Compute or I/O node was a single ASIC with associated DRAM memory chips. The ASIC integrated two 700 MHz PowerPC 440 embedded processors, each with a double-pipeline-double-precision Floating Point Unit (FPU), a cache sub-system with built-in DRAM controller and the logic to support multiple communication sub-systems. The dual FPUs gave each Blue Gene/L node a theoretical peak performance of 5.6 GFLOPS (gigaFLOPS). The two CPUs were not cache coherent with one another.

Compute nodes were packaged two per compute card, with 16 compute cards plus up to 2 I/O nodes per node board. There were 32 node boards per cabinet/rack.[17] By the integration of all essential sub-systems on a single chip, and the use of low-power logic, each Compute or I/O node dissipated low power (about 17 watts, including DRAMs). This allowed aggressive packaging of up to 1024 compute nodes, plus additional I/O nodes, in a standard 19-inch rack, within reasonable limits of electrical power supply and air cooling. The performance metrics, in terms of FLOPS per watt, FLOPS per m2 of floorspace and FLOPS per unit cost, allowed scaling up to very high performance. With so many nodes, component failures were inevitable. The system was able to electrically isolate faulty components, down to a granularity of half a rack (512 compute nodes), to allow the machine to continue to run.

Each Blue Gene/L node was attached to three parallel communications networks: a 3D toroidal network for peer-to-peer communication between compute nodes, a collective network for collective communication (broadcasts and reduce operations), and a global interrupt network for fast barriers. The I/O nodes, which run the Linux operating system, provided communication to storage and external hosts via an Ethernet network. The I/O nodes handled filesystem operations on behalf of the compute nodes. Finally, a separate and private Ethernet network provided access to any node for configuration, booting and diagnostics. To allow multiple programs to run concurrently, a Blue Gene/L system could be partitioned into electronically isolated sets of nodes. The number of nodes in a partition had to be a positive integer power of 2, with at least 25 = 32 nodes. To run a program on Blue Gene/L, a partition of the computer was first to be reserved. The program was then loaded and run on all the nodes within the partition, and no other program could access nodes within the partition while it was in use. Upon completion, the partition nodes were released for future programs to use.

Blue Gene/L compute nodes used a minimal operating system supporting a single user program. Only a subset of POSIX calls was supported, and only one process could run at a time on node in co-processor mode—or one process per CPU in virtual mode. Programmers needed to implement green threads in order to simulate local concurrency. Application development was usually performed in C, C++, or Fortran using MPI for communication. However, some scripting languages such as Ruby[18] and Python[19] have been ported to the compute nodes.

Blue Gene/P

A Blue Gene/P node card
A schematic overview of a Blue Gene/P supercomputer

In June 2007, IBM unveiled Blue Gene/P, the second generation of the Blue Gene series of supercomputers and designed through a collaboration that included IBM, LLNL, and Argonne National Laboratory's Leadership Computing Facility.[20]

Design

The design of Blue Gene/P is a technology evolution from Blue Gene/L. Each Blue Gene/P Compute chip contains four PowerPC 450 processor cores, running at 850 MHz. The cores are cache coherent and the chip can operate as a 4-way symmetric multiprocessor (SMP). The memory subsystem on the chip consists of small private L2 caches, a central shared 8 MB L3 cache, and dual DDR2 memory controllers. The chip also integrates the logic for node-to-node communication, using the same network topologies as Blue Gene/L, but at more than twice the bandwidth. A compute card contains a Blue Gene/P chip with 2 or 4 GB DRAM, comprising a "compute node". A single compute node has a peak performance of 13.6 GFLOPS. 32 Compute cards are plugged into an air-cooled node board. A rack contains 32 node boards (thus 1024 nodes, 4096 processor cores).[21] By using many small, low-power, densely packaged chips, Blue Gene/P exceeded the power efficiency of other supercomputers of its generation, and at 371 MFLOPS/W Blue Gene/P installations ranked at or near the top of the Green500 lists in 2007-2008.[2]

Installations

The following is an incomplete list of Blue Gene/P installations. Per November 2009, the TOP500 list contained 15 Blue Gene/P installations of 2-racks (2048 nodes, 8192 processor cores, 23.86 TFLOPS Linpack) and larger.[1]

Applications

Blue Gene/Q

The IBM Blue Gene/Q installed at Argonne National Laboratory, near Chicago, Illinois.

The third supercomputer design in the Blue Gene series, Blue Gene/Q has a peak performance of 20 Petaflops,[37] reaching LINPACK benchmarks performance of 17 Petaflops. Blue Gene/Q continues to expand and enhance the Blue Gene/L and /P architectures.

Design

The Blue Gene/Q Compute chip is an 18 core chip. The 64-bit A2 processor cores are 4-way simultaneously multithreaded, and run at 1.6 GHz. Each processor core has a SIMD Quad-vector double precision floating point unit (IBM QPX). 16 Processor cores are used for computing, and a 17th core for operating system assist functions such as interrupts, asynchronous I/O, MPI pacing and RAS. The 18th core is used as a redundant spare, used to increase manufacturing yield. The spared-out core is shut down in functional operation. The processor cores are linked by a crossbar switch to a 32 MB eDRAM L2 cache, operating at half core speed. The L2 cache is multi-versioned, supporting transactional memory and speculative execution, and has hardware support for atomic operations.[38] L2 cache misses are handled by two built-in DDR3 memory controllers running at 1.33 GHz. The chip also integrates logic for chip-to-chip communications in a 5D torus configuration, with 2GB/s chip-to-chip links. The Blue Gene/Q chip is manufactured on IBM's copper SOI process at 45 nm. It delivers a peak performance of 204.8 GFLOPS at 1.6 GHz, drawing about 55 watts. The chip measures 19×19 mm (359.5 mm²) and comprises 1.47 billion transistors. The chip is mounted on a compute card along with 16 GB DDR3 DRAM (i.e., 1 GB for each user processor core).[39]

A Q32[40] compute drawer contains 32 compute cards, each water cooled.[41] A "midplane" (crate) contains 16 Q32 compute drawers for a total of 512 compute nodes, electrically interconnected in a 5D torus configuration (4x4x4x4x2). Beyond the midplane level, all connections are optical. Racks have two midplanes, thus 32 compute drawers, for a total of 1024 compute nodes, 16,384 user cores and 16 TB RAM.[41]

Separate I/O drawers, placed at the top of a rack or in a separate rack, are air cooled and contain 8 compute cards and 8 PCIe expansion slots for Infiniband or 10 Gigabit Ethernet networking.[41]

Performance

At the time of the Blue Gene/Q system announcement in November 2011, an initial 4-rack Blue Gene/Q system (4096 nodes, 65536 user processor cores) achieved #17 in the TOP500 list[1] with 677.1 TeraFLOPS Linpack, outperforming the original 2007 104-rack BlueGene/L installation described above. The same 4-rack system achieved the top position in the Graph500 list[3] with over 250 GTEPS (giga traversed edges per second). Blue Gene/Q systems also topped the Green500 list of most energy efficient supercomputers with up to 2.1 GFLOPS/W.[2]

In June 2012, Blue Gene/Q installations took the top positions in all three lists: TOP500,[1] Graph500 [3] and Green500.[2]

Installations

The following is an incomplete list of Blue Gene/Q installations. Per June 2012, the TOP500 list contained 20 Blue Gene/Q installations of 1/2-rack (512 nodes, 8192 processor cores, 86.35 TFLOPS Linpack) and larger.[1] At a (size-independent) power efficiency of about 2.1 GFLOPS/W, all these systems also populated the top of the June 2012 Green 500 list.[2]

Applications

Record-breaking science applications have been run on the BG/Q, the first to cross 10 petaflops of sustained performance. The cosmology simulation framework HACC achieved almost 14 petaflops with a 3.6 trillion particle benchmark run,[60] while the Cardioid code,[61][62] which models the electrophysiology of the human heart, achieved nearly 12 petaflops with a near real-time simulation, both on Sequoia. A fully compressible flow solver has also achieved 14.4 PFLOP/s (originally 11 PFLOP/s) on Sequoia, 72% of the machine's nominal peak performance.[63]

See also

Notes and references

  1. 1 2 3 4 5 6 7 8 9 "The Top500 List".
  2. 1 2 3 4 5 "The Green500 List".
  3. 1 2 3 "The Graph500 List". Archived from the original on 2011-12-27.
  4. Harris, Mark (September 18, 2009). "Obama honours IBM supercomputer". Techradar. Retrieved 2009-09-18.
  5. Supercomputing Strategy Shifts in a World Without BlueGene
  6. IBM to Build DoE's Next-Gen Coral Supercomputers
  7. "Blue Gene: A Vision for Protein Science using a Petaflop Supercomputer" (PDF). IBM Systems Journal, Special Issue on Deep Computing for the Life Sciences. 40 (2).
  8. "A Talk with the Brain behind Blue Gene", BusinessWeek, November 6, 2001.
  9. "Lawrence Livermore National Laboratory: BlueGene/L".
  10. hpcwire.com Archived September 28, 2007, at the Wayback Machine.
  11. SC06
  12. hpcchallenge.org
  13. "Mouse brain simulated on computer". BBC News. April 27, 2007. Archived from the original on 2007-05-25.
  14. IBM100 - Blue Gene
  15. "Blue Gene". IBM Journal of Research and Development. 49 (2/3). 2005.
  16. Bluegene/L Configuration https://asc.llnl.gov/computing_resources/bluegenel/configuration.html
  17. ece.iastate.edu Archived April 29, 2007, at the Wayback Machine.
  18. William Scullin (March 12, 2011). Python for High Performance Computing. Atlanta, GA.
  19. "IBM Triples Performance of World's Fastest, Most Energy-Efficient Supercomputer". 2007-06-27. Retrieved 2011-12-24.
  20. "Overview of the IBM Blue Gene/P project". IBM Journal of Research and Development. 52: 199–220. Jan 2008. doi:10.1147/rd.521.0199.
  21. "Supercomputing: Jülich Amongst World Leaders Again". IDG News Service. 2007-11-12.
  22. "IBM Press room - 2009-02-10 New IBM Petaflop Supercomputer at German Forschungszentrum Juelich to Be Europe's Most Powerful". www-03.ibm.com. 2009-02-10. Retrieved 2011-03-11.
  23. "Argonne's Supercomputer Named World’s Fastest for Open Science, Third Overall"
  24. "Rice University, IBM partner to bring the first Blue Gene supercomputer to Texas, March 2012".
  25. Вече си имаме и суперкомпютър, Dir.bg, 9 September 2008
  26. "IBM Press room - 2010-02-11 IBM to Collaborate with Leading Australian Institutions to Push the Boundaries of Medical Research - Australia". 03.ibm.com. 2010-02-11. Retrieved 2011-03-11.
  27. http://www.informationweek.com/hardware/supercomputers/rutgers-gets-big-data-weapon-in-ibm-supe/232700313
  28. "University of Rochester and IBM Expand Partnership in Pursuit of New Frontiers in Health". University of Rochester Medical Center. May 11, 2012. Archived from the original on 2012-05-11.
  29. "IBM and Universiti Brunei Darussalam to Collaborate on Climate Modeling Research". IBM News Room. Retrieved 18 October 2012.
  30. http://www.philstar.com/headlines/2014/04/15/1312838/dosts-supercomputer-scientists-now-operational
  31. "Topalov training with super computer Blue Gene P". Chessdom. Retrieved 21 May 2010.
  32. Kaku, Michio. Physics of the Future (New York: Doubleday, 2011), 91.
  33. "Project Kittyhawk: A Global-Scale Computer".
  34. Project Kittyhawk: building a global-scale computer
  35. "Rutgers-led Experts Assemble Globe-Spanning Supercomputer Cloud". http://news.rutgers.edu. 2011-07-06. Archived from the original on 2011-11-10. Retrieved 2011-12-24. External link in |publisher= (help)
  36. "IBM announces 20-petaflops supercomputer". Kurzweil. 18 November 2011. Retrieved 13 November 2012. IBM has announced the Blue Gene/Q supercomputer, with peak performance of 20 petaflops
  37. "Memory Speculation of the Blue Gene/Q Compute Chip". Retrieved 2011-12-23.
  38. "The Blue Gene/Q Compute chip" (PDF). Retrieved 2011-12-23.
  39. IBM Blue Gene/Q supercomputer delivers petascale computing for high-performance computing applications
  40. 1 2 3 "IBM uncloaks 20 petaflops BlueGene/Q super". The Register. 2010-11-22. Retrieved 2010-11-25.
  41. Feldman, Michael (2009-02-03). "Lawrence Livermore Prepares for 20 Petaflop Blue Gene/Q". HPCwire. Retrieved 2011-03-11.
  42. B Johnston, Donald (2012-06-18). "NNSA's Sequoia supercomputer ranked as world's fastest". Retrieved 2012-06-23.
  43. TOP500 Press Release Archived 2012-06-24 at the Wayback Machine.
  44. MIRA: World's fastest supercomputer
  45. ANL's Mira Homepage
  46. http://hpcinnovationcenter.llnl.gov/
  47. https://www.llnl.gov/news/newsreleases/2013/Jun/NR-13-06-05.html
  48. "Archived copy". Archived from the original on 2013-10-30. Retrieved 2013-05-13.
  49. http://news.rpi.edu/content/2014/08/04/amos-ranks-43rd-list-world%E2%80%99s-top-500-supercomputers
  50. http://themelbourneengineer.eng.unimelb.edu.au/2012/02/worlds-greenest-computer-comes-to-melbourne/
  51. Victorian Life Sciences Computation Initiative
  52. VLSCI's Computer & Software Configuration
  53. http://www.rochester.edu/news/show.php?id=4192
  54. http://www.circ.rochester.edu/resources.html
  55. http://bluegene.epfl.ch
  56. http://www.cadmos.org
  57. "A*STAR Computational Resource Centre". www.acrc.a-star.edu.sg. Retrieved 2016-08-24.
  58. S. Habib; V. Morozov; H. Finkel; A. Pope; K. Heitmann; K. Kumaran; T. Peterka; J. Insley; D. Daniel; P. Fasel; N. Frontiere & Z. Lukic. "The Universe at Extreme Scale: Multi-Petaflop Sky Simulation on the BG/Q". arXiv:1211.4864Freely accessible.
  59. "Cardioid Cardiac Modeling Project".
  60. "Venturing into the Heart of High-Performance Computing Simulations".
  61. "Cloud cavitation collapse". Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13. doi:10.1145/2503210.2504565.
Records
Preceded by
NEC Earth Simulator
35.86 teraflops
World's most powerful supercomputer
(Blue Gene/L)

November 2004 – November 2007
Succeeded by
IBM Roadrunner
1.026 petaflops
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.