121–125 of 125 posts
Earlier quoted context omitted.
What's this all look like without an atmosphere?
Worse, heat dissipation is a major constraint for spacecraft and satellites because you can only radiate heat away as infrared photons.
Earlier quoted context omitted.
> It always made me wonder why liquid cooling wasn't more of a thing for datacenters. Liquid cooling is almost a defacto-standard in data centers in the HPC world. The Top of the TOP500 machines are all liquid cooled. Not by choice, but due to physics constraints. There is a big gap in power density between the HPC world and the usual datacenter-commodity-hardware world. Commodity DS are designed with the assumption…
I’m a bit skeptical about the claim that the top of the TOP500 are all liquid cooled due to physical constraints. (Where the “physics” here seems to mean the density and property of air cool in general but ignoring the environment of the machine such as the ambient weather.) The one data point I know well is NERSC, and has been air cooled until relatively recently. Part of the success of air cool in the past was the…
- Frontier: Liquid cooled
- Aurora: liquid cooled
- Eagle: liquid cooled
- Fugaku: liquid cooled
- Lumi: Liquid cooled with heat recycling.
The need for liquid cooled arrived with the usage of accelerators that bumped significantly the power density.
That said, Fugaku which is not accelerator driven is also liquid cooled today.
I am surprised to learn that NERSC was air cooled until recently. Most of the BGQ machines, that were dominating the top500 10 years ago, were already liquid cooled.
Earlier quoted context omitted.
> It always made me wonder why liquid cooling wasn't more of a thing for datacenters. Liquid cooling is almost a defacto-standard in data centers in the HPC world. The Top of the TOP500 machines are all liquid cooled. Not by choice, but due to physics constraints. There is a big gap in power density between the HPC world and the usual datacenter-commodity-hardware world. Commodity DS are designed with the assumption…
> we attempted to install a medium size HPC cluster in a well-known commerical Datacenter and network provider. The commercial of the DS almost fall from his chair when we announced the power requirements. Heh. We tried it too. They didn’t believe that a single node used their entire rack’s budget at first.
Also, this involves a UK university where it would takes forever to upgrade the power delivery to the building/floor/room. There's no planned upgrade whatsoever so people just need to make do with what we have.
Earlier quoted context omitted.
I’m a bit skeptical about the claim that the top of the TOP500 are all liquid cooled due to physical constraints. (Where the “physics” here seems to mean the density and property of air cool in general but ignoring the environment of the machine such as the ambient weather.) The one data point I know well is NERSC, and has been air cooled until relatively recently. Part of the success of air cool in the past was the…
If you take the top5 of the top500 (01/2024): - Frontier: Liquid cooled - Aurora: liquid cooled - Eagle: liquid cooled - Fugaku: liquid cooled - Lumi: Liquid cooled with heat recycling. The need for liquid cooled arrived with the usage of accelerators that bumped significantly the power density. That said, Fugaku which is not accelerator driven is also liquid cooled today. I am surprised to learn that NERSC was air c…
But perhaps you're right that the physics is the ultimate factor here: what comes in must goes out. With order of magnitude scale of increase in power coming in, water seems to be the most effective and cheap entity to absorb those heat and be transported outside the floor very efficiently.
The hardware / datacenter side of this is equally fascinating. I used to work in AWS, but on the software / services side of things. But now and then, we would crash some talks from the datacenter folks. One key relevation for me was that increasing compute power in DCs is primarily a thermodynamics problem than actual computing. The nodes have become so dense that shipping power in and shipping heat out, with all ki…
It always made me wonder why liquid cooling wasn't more of a thing for datacenters. Water has a massive amount of thermal capacity and can quickly and in bulk be cooled to optimal temperatures. You'd probably still need fans and AC to dissipate heat of non-liquid cooled parts, but for the big energy items like CPUs and GPUs/compute engines, you could ship out huge amounts of heat fairly quickly and directly. I guess…
Problem is that’s how you sell 1-3 racks not an entire room of them.
Ethylene lines might be more practical, easier to wire into existing systems.