Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How oxide cuts data center power consumption in half

oxide.computer

Post reply on HN
Print view View original

21–30 of 185 posts

Re: How oxide cuts data center power consumption in half

#21

Post by farawayea »

renewiltord wrote: post #10

What I don't get is why tie to such an ancient platform. AMD Milan is my home lab. The new 9004 Epycs are so much better on power efficiency. I'm sure they've done their market research and the gains must be so significant. We used to have a few petabytes and tens of thousands of cores almost ten years ago and it's crazy how much higher data and compute density you can get with modern 30 TiB disks and Epyc 9654s. 100…

They've built this a while ago. A hardware refresh takes time. The good news is that they may be able to upgrade the existing equipment with newer sleds.

Re: How oxide cuts data center power consumption in half

#23

Post by bigfatkitten »

PreInternet01 wrote: post #12
yjftsjthsd-h wrote: post #9

Earlier quoted context omitted.

> it's just that Rewrite It In (Safe) Rust is unlikely to help with that... I didn't see any mention of Rust in the article?

[flagged]

They wrote their own BMC and various other bits and pieces in Rust. That's an extremely tiny part of the whole picture.

Re: How oxide cuts data center power consumption in half

#24

Post by walrus01 »

zcw100 wrote: post #22

I believe the telco’s did dc power for years so I don’t think this anything new. Any old hands out there want to school us on how it was done in the old days?

big ass rectifiers

big ass solid copper busbars

huge gauge copper cables going around a central office (google "telcoflex IV")

big DC breaker/fuse panels

specialized dc fuse panels for power distribution at the top of racks, using little tiny fuses

100% overhead steel ladder rack type cable trays, since your typical telco CO was never a raised floor type environment (UNLIKE legacy 1960s/1970s mainframe computer rooms), so all the power was kept accessible by a team of people working on stepladders.

The same general thing continues today in serious telco/ISP operations, with tech features to bring it into the modern era. The rectifiers are modular now, and there's also rectiverters. Monitoring is much better. People are moving rapidly away from wet cell 2V lead acid battery banks and AGM sealed lead acid stuff to LiFePo4 battery systems.

DC fuse panels can come with network-based monitoring, ability to turn on/off devices remotely.

equipment is a whole lot less power hungry now, a telco CO that has decommed a 5ESS will find itself with a ton of empty thermal and power budget.

when I say serious telco stuff is a lot less power hungry, it's by huge margins. randomly chosen example of radio transport equipment. For instance back in the day a powerful, very expensive point to point microwave radio system might be a full 42U rack, 800W in load, with waveguide going out to antennas on a roof. It would carry one, two or three DS3 equivalent of capacity (45 Mbps each).

now, that same telco might have a radio on its CO roof in the same microwave bands that is 1.3 Gbps FDD capacity, pure ethernet with a SFP+ fiber interface built into it, and the whole radio is a 40W electrical load. The radio is mounted directly on the antenna with some UV/IR resistant weatherproof 16 gauge DC power cable running down into the CO and plugged into a fuse panel.

Re: How oxide cuts data center power consumption in half

#25

Post by zamalek »

unsnap_biceps wrote: post #14

> When we started Oxide, the DC bus bar stood as one of the most glaring differences between the rack-scale machines at the hyperscalers and the rack-and-stack servers that the rest of the market was stuck with. That a relatively simple piece of copper was unavailable to commercial buyers It seems that 0xide was founded in 2019 and Open Compute Project had been specifying dc bus bars for 6 years at that point. People…

It's normally incredibly difficult for employees to disrupt at massive companies that would be the type which runs a data center. Disruption usually enters the corp in a sales deck, much like the one Oxide would have.

It's stupid, but that's why we all have jobs.

Re: How oxide cuts data center power consumption in half

#26

Post by farawayea »

Their tech may be more than adequate today. Bigger businesses may not buy from a small startup company. They expect a lot more. Illumos is a less popular OS. It wouldn't be the first choice for the OS I'd rely on. Who writes the security mitigations for speculative execution bugs? Who patches CVEs in the shipped software which doesn't use Rust?

Re: How oxide cuts data center power consumption in half

#27

Post by autoexecbat »

AceJohnny2 wrote: post #15
AceJohnny2 wrote: post #11

Earlier quoted context omitted.

I will add that consumer macOS is a piss-poor server OS. At one point, for many years, it would just sometimes fail to `exec()` a process. This would manifest as a random failure on our build farm about once/twice a month. (This would manifest as "/bin/sh: fail to exec binary file" because the error type from the kernel would have the libc fall back to trying to run the binary as a script, as normal for a Unix, but i…

> as normal for a Unix veering offtopic, did you know macOS is a certified Unix? https://www.opengroup.org/openbrand/register/brand3581.htm As I recall, Apple advertised macOS as a Unix without such certification, got sued, and then scrambled to implement the required features to get certification as a result. Here's the story as told by the lead engineer of the project: https://www.quora.com/What-goes-into-making-an…

and Windows used to be certified for posix, but none of that matters theses days if it's not bug-compatible with Linux

Re: How oxide cuts data center power consumption in half

#28

Post by rincebrain »

AceJohnny2 wrote: post #11
rincebrain wrote: post #7

Earlier quoted context omitted.

I don't think they'd admit much about it even if they had one internally, both because Apple isn't known for their openness about many things, and because they already exited the dedicated server hardware business years ago, so I think they're likely averse to re-entering it without very strong evidence that it would be beneficial for more than a brief period. In particular, while I'd enjoy such a device, Apple's who…

I will add that consumer macOS is a piss-poor server OS. At one point, for many years, it would just sometimes fail to `exec()` a process. This would manifest as a random failure on our build farm about once/twice a month. (This would manifest as "/bin/sh: fail to exec binary file" because the error type from the kernel would have the libc fall back to trying to run the binary as a script, as normal for a Unix, but i…

Did that ever get fixed? That...seems like a pretty critical problem.

Re: How oxide cuts data center power consumption in half

#29

Post by formerly_proven »

walrus01 wrote: post #19
shivak wrote: post #17

> > The power shelf distributes DC power up and down the rack via a bus bar. This eliminates the 70 total AC power supplies found in an equivalent legacy server rack within 32 servers, two top-of-rack switches, and one out-of-band switch, each with two AC power supplies This creates a single point of failure, trading robustness for efficiency. There's nothing wrong with that, but software/ops might have to accommodat…

The whole thing with eliminating 70 discrete 1U server size AC-to-DC power supplies is nothing new. It's the same general concept as the power distribution unit in the center of an open compute platform rack design from 10+ years ago. Everyone who's doing serious datacenter stuff at scale knows that one of the absolute least efficient, labor intensive and cabling intensive/annoying ways of powering stuff is to have s…

That’s one reason why 2U4N systems are kinda popular. 1/4 the cabling in legacy infrastructure.

Re: How oxide cuts data center power consumption in half

#30

Post by einpoklum »

> How can organizations reduce power consumption and corresponding carbon emissions?

Stop running so much useless stuff.

Also maybe ARM over x86_64 and similar power-efficiency-oriented hardware.

Rack-level system design, or at least power & cooling design, is certainly also a reasonable thing to do. But standardization is probably important here, rather than some bespoke solution which only one provider/supplier offers.

> How can organizations keep pace with AI innovation as existing data centers run out of available power?

Waste less energy on LLM chatbots?

Post reply on HN