Server immersion cooling

Example of an immersed computing® system

Example of server immersion cooling of one server

Immersion cooling with large open baths

Server immersion cooling is a computer cooling practice by which computer components or servers are submerged in a thermally conductive dielectric liquid. Common dielectrics which are suitable for immersion cooling are either oil or chemically based.

Server immersion cooling has the potential of becoming a popular server cooling solution for green data centers,^[1] as it allows them to drastically reduce their energy load, regardless of their Power Usage Effectiveness (PUE). Multiple relevant brands like Intel and Facebook have already validated the advantages of submerging servers.^[2][3]

Servers and other IT hardware cooled by immersion cooling do not require fans, thus these are removed. thermal paste needs to be replaced with a different compound to avoid it from dissolving in the oil. Indium foil^[4] is commonly used as a replacement material. The liquids used in immersion cooling usually operate at temperatures between 35 to 60 °C (95 to 140 °F), this allows data centers to use much more efficient evaporative or adiabatic cooling towers^[5] instead of chiller-based air cooling.

Current commercial applications for immersion cooling range from data center oriented solutions for commodity server cooling,^[6][7] server clusters, HPCC applications^[8] and Bitcoin Mining.^[9] Mainstream cloud based architectures are now added to this list due to the launch of Immersed Computing®^[10] in March, 2017, which addresses specific demands for tier-4 datacentres and normal whitespace.

History

Although still rarely used for cooling of IT hardware, the usage of mineral or vegetable-based oil for cooling large power distribution components such as transformers and industrial machinery is a common practice since the beginning of the 1900s^[11]

In 1925, the first patent was filed for using oil as an insulation material for transformers.

In 1968, the first patent was filed for "an immersion cooling system for modularly packaged components" by "International Business Machines Corporation".

This patent was followed by several other patents related to the immersion cooling of server systems. Some extreme density supercomputers such as the Cray-2 (released in 1985) and Cray T90 (released in 1995) used large liquid-to-chilled liquid heat exchangers and single or two-phase immersion cooling oils for heat removal^[12]

A different industry where immersion cooling is applied to electronic components is deep sea research where Remotely Operated unmanned Vehicles with electronic equipment are filled with oil to protect them from corrosion in seawater and being crushed by the extreme pressure exerted on the ROV while working in the deep sea. Although this is not meant to solve any cooling issues, the applied technology is the same.

The technology was until recently only applied in a limited number of special computing projects. Since the high increase in global energy consumption by datacenters, there is now a focus on energy efficiency which has driven immersion into a second life. Since 2005, several companies tried to claim patents based on the open bath approach with external pump installations. Most, if not all of these claims have been denied on the basis that IBM had already patented the technology.

Server Immersion Cooling techniques

Open or Semi-Open Bath Immersion Cooling

Open or semi-open bath immersion cooling is a data center cooling technique that implies fully submerging servers in dielectric coolant fluid. It is called open or semi-open bath because servers are placed side-by-side in large tanks that function as baths. These baths operate at atmospheric pressure and allow the coolant fluid to be moved through the hardware components or servers submerged in it.^[13]

Open Bath Immersion Cooling uses single-phase dielectric liquids. These liquids always remain in liquid state while operating. They never boil or freeze. The oil is usually pumped to an external heat exchanger where it is cooled thanks to heat exchange with a cooler water-circuit. A different approach uses natural convection as the driving force to circulate the oil^[14] through the servers. and a water cooled heat exchanger. This technique uses “open baths“ as there’s little (or no) risk of the coolant evaporating.

In semi-open bath Immersion Cooling, a two-phase chemically engineered liquid is the working fluid. It literally boils thanks to its low boiling point^[15][16] and thus exists in both a liquid and gas phase. The system takes advantage of a concept known as “latent heat” which is the heat (thermal energy) required to change the phase of a fluid (in this case two-phase dielectric mineral oil). The oil is only cooled by boiling and thus remains at the boiling point (“saturation temperature”). Energy transferred from the servers into the two-phase oil will cause a portion of it to boil off into a gas (this is the second phase of the oil). The gas rises above the liquid oil level where it contacts a condenser which is cooler than the saturation temperature. This causes the vapoured oil to condense back into a liquid form and fall (rain) back into the bath.^[17]

The term "semi-open bath" means, that when the system operates, it is sealed to avoid the evaporation or the coolant.

Sealed Server Immersion Cooling

A different approach to open or semi-open bath immersion cooling are sealed server immersion cooling solutions, where servers are specially built in a liquid-tight casing. The dielectric liquid (oil or chemically based) is circulated inside or pumped through each server to collect heat from the components. The heat is then taken to a backplane in the rack which is cooled by a water circuit.^[18]

The main advantage of this approach is that servers don't require to be placed in horizontal baths. The most common disadvantage is that not any hardware can be used as the vendor defines the hardware specs of the sealed servers.^[19]

References