Cooling Data Centers: From Blowing Air to Taking a Bath

Traditional machine rooms cool themselves by blowing air. That approach has worked for thirty years, and AI training racks have just pushed it to its physical limit.

Image: BalticServers data center (BalticServers.com), CC BY-SA 3.0, via Wikimedia Commons.

The physical ceiling of air cooling

Air cooling is conceptually simple: push cold air through the rack and let it carry the heat away. The problem is that air has both a low specific heat capacity and low thermal conductivity, so a given volume of it moves far less heat than the same volume of liquid.

• Ten years ago, 5 to 10 kilowatts per rack counted as high density.
• Today, general-purpose racks commonly run 10 to 20 kilowatts.
• AI training racks reach 40 to 130 kilowatts.

Air cooling realistically tops out around 20 to 30 kilowatts per rack. Beyond that there are only two moves: spin the fans to their maximum, which explodes noise and fan power, or drop the supply air temperature further, which makes refrigeration more expensive. Both are dead ends. PUE, the ratio of total power to IT power, is already down to 1.1 to 1.2 in hyperscale sites, and what remains is largely a cooling problem.

Direct-to-chip liquid cooling

Direct-to-chip cooling presses a metal cold plate onto the CPU or GPU and runs liquid through microchannels inside the plate. Its biggest advantage is a small blast radius: the building does not need rebuilding, only the inside of the rack and a secondary loop.

The costs are equally clear:

• Quick disconnects, hoses, leak detection and maintenance procedures all have to be redesigned. Operations shift from swapping servers to swapping plumbing.
• Coolant usually runs at 40 to 60 degrees Celsius, a far higher grade of waste heat than air cooling produces. That is what finally makes heat reuse worthwhile.
• It turns cooling from a facility problem into a rack and chip-packaging problem.

Immersion cooling

The more radical option is to submerge the whole server in a non-conductive liquid.

Image: Single-server immersion cooling (Submer), CC BY-SA 4.0, via Wikimedia Commons.

There are two variants. Single-phase systems pump liquid through an external heat exchanger. Two-phase systems let the liquid boil directly on the chip surface and carry heat away as latent heat, which is the most efficient, and also the most sensitive to sealing and fluid cost.

Image: Asperitas AIC24 immersion system (Rolf Brink), CC BY-SA 4.0, via Wikimedia Commons.

The upside is that fans all but disappear, machine-room noise collapses, and PUE can reach 1.03 to 1.05. The friction is maintenance: servers come out of the tank sticky and heavy, mechanical and optical parts such as drives and transceivers need their fluid compatibility checked one by one, and the fluid itself is expensive and needs filtering and replacement.

Waste heat is not waste

The most overlooked point in cooling debates is that the heat is a resource. A number of Nordic projects already feed data center waste heat into district heating, greenhouses or pools. Immersion cooling makes this genuinely practical because its return water is hot enough, 60 to 70 degrees Celsius, to feed a heat pump or exchanger directly. Air cooling exhausts at thirty-something degrees, which is worth far less.

Where I land

Cooling is turning from a facilities problem into a chip problem. Cold plates, packaging, racks and secondary loops will be designed together rather than each team owning one segment.

For a small self-built machine room, air cooling with contained hot and cold aisles stays the right answer for a long time. Liquid cooling is not for everyone. The dividing line is power density: below 30 kilowatts per rack you do not need liquid, and above it you should stop forcing air. Choose by density, not by fashion.