Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How oxide cuts data center power consumption in half

oxide.computer

Post reply on HN
Print view View original

61–70 of 185 posts

Re: How oxide cuts data center power consumption in half

#61

Post by danpalmer »

ZeroCool2u wrote: post #52

If any Oxide staff are here, I'm just curious, is BlueSky a customer? Seems like it would fit well with their on-prem setup.

Not Oxide or Bluesky, but firstly I'd suggest that asking the company about their customers is unlikely to get a response, most companies don't disclose their customers. Secondly, Bluesky have been growing quickly, I can only assume their hardware is too, and that means long lead time products like an Oxide rack aren't going to work, especially when you can have an off the shelf machine from Dell delivered in a few days.

Re: How oxide cuts data center power consumption in half

#62

Post by sidewndr46 »

gruez wrote: post #58
sidewndr46 wrote: post #36

Earlier quoted context omitted.

This isn't even remotely close. Unless all 32 servers have redundant AC power feeds present, you've traded one single point of failure for another single point of failure. In the event that all 32 servers had redundant AC power feeds, you could just install a pair of redundant DC power feeds.

>Unless all 32 servers have redundant AC power feeds present, you've traded one single point of failure for another single point of failure. Is this not standard? I vaguely remember that rack severs typically have two PSUs for this reason.

you could have 15 PSUs in a server. It doesn't mean they have redundant power feeds

Re: How oxide cuts data center power consumption in half

#63

Post by hn-throw »

lillecarl wrote: post #46
hn-throw wrote: post #40

Earlier quoted context omitted.

Let's say your high quality supply's yearly failure rate is 100 times less than the cheap ones The probability of at least a single failure is 1-(1-r)^70. This is quite high even w/out considering the higher quality of the one supply. The probability of all 70 going down is r^70 which is absurdly low. Let's say r = 0.05 or one failed supply every 20 in a year. 1-(1-r)^70 = 97% r^70 The high quality supply has r = 0.0…

Yeah but the failure rate of an analog piece of copper is pretty low, it'll keep being copper unless you do stupid things. You'll have multiple power supplies provide power on the same piece of copper

TL/DR, isnt there a single, shared, DC supply that supplies said piece of copper? Presumably connected to mains?

Or are the running on SOFCs?

Re: How oxide cuts data center power consumption in half

#64

Post by eaasen »

immibis wrote: post #41
jsolson wrote: post #35

Earlier quoted context omitted.

The bus bar itself is an SPoF, but it's also just dumb copper. That doesn't mean that nothing can go wrong, but it's pretty far into the tail of the failure distribution. The power shelf that keeps the busbar fed will have multiple rectifiers, often with at least N+1 redundancy so that you can have a rectifier fail and swap it without the rack itself failing. Similar things apply to the battery shelves.

It's also plausible to have multiple power supplies feeding the same bus bar in parallel (if they're designed to support this) e.g. one at each end of a row.

This is how our rack works (Oxide employee). In each power shelf, there are 6 power supplies and only 5 need to be functional to run at full load. If you want even more redundancy, you can use both power shelves with independent power feeds to each so even if you lose a feed, the rack still has 5+1 redundant power supplies.

Re: How oxide cuts data center power consumption in half

#65

Post by andrewjf »

hnthrowaway0328 wrote: post #49
zamalek wrote: post #25

Earlier quoted context omitted.

It's normally incredibly difficult for employees to disrupt at massive companies that would be the type which runs a data center. Disruption usually enters the corp in a sales deck, much like the one Oxide would have. It's stupid, but that's why we all have jobs.

I think engineers should be more forceful to lead their own visions instead being led by accountants and lawyers. After engineers have the power of implementation and de-implementstion. They need to step into dirty politics and bend other people's views. It's either theirs or ours. Win-win is a fallacy.

Being able to navigate this is what differentiates a very senior IC (principal, distinguished, etc) and random employees.

Re: How oxide cuts data center power consumption in half

#66

Post by eaasen »

lillecarl wrote: post #50
toomuchtodo wrote: post #37

Earlier quoted context omitted.

This. If you want on prem cloud infra without having to roll it yourself, Oxide is the solution. (no affiliation, just a fan)

If you want on prem infra in exactly the shape and form Oxide delivers* I've read and understood from Joyent and SmartOS that they believe fault tolerant block devices / filesystems is the wrong abstraction, your software should handle losing storage.

We do not put the onus on customers to tolerate data loss. Our storage is redundant and spread through the rack so that if you lose drives or even an entire computer, your data is still safe. https://oxide.computer/product/storage

Re: How oxide cuts data center power consumption in half

#67

Post by jorams »

AceJohnny2 wrote: post #15
AceJohnny2 wrote: post #11

Earlier quoted context omitted.

I will add that consumer macOS is a piss-poor server OS. At one point, for many years, it would just sometimes fail to `exec()` a process. This would manifest as a random failure on our build farm about once/twice a month. (This would manifest as "/bin/sh: fail to exec binary file" because the error type from the kernel would have the libc fall back to trying to run the binary as a script, as normal for a Unix, but i…

> as normal for a Unix veering offtopic, did you know macOS is a certified Unix? https://www.opengroup.org/openbrand/register/brand3581.htm As I recall, Apple advertised macOS as a Unix without such certification, got sued, and then scrambled to implement the required features to get certification as a result. Here's the story as told by the lead engineer of the project: https://www.quora.com/What-goes-into-making-an…

This comes up rather often, and on the last significant post about it I saw on HN someone pointed out that the certification is kind of meaningless[1]. macOS poll(2) is not Unix-compliant, hasn't been since forever, yet every new version of macOS gets certified regardless.

[1]: https://news.ycombinator.com/item?id=41823078

Re: How oxide cuts data center power consumption in half

#68

Post by thfuran »

glitchcrab wrote: post #60
gruez wrote: post #58

Earlier quoted context omitted.

>Unless all 32 servers have redundant AC power feeds present, you've traded one single point of failure for another single point of failure. Is this not standard? I vaguely remember that rack severs typically have two PSUs for this reason.

It's highly dependent on the individual server model and quite often how you spec it too. Most 1U Dell machines I worked with in the past only had a single slot for a PSU, whereas the beefier 2U (and above) machines generally came with 2 PSUs.

But 2 PSUs plugged into the same AC supply still have a single point of failure.

Re: How oxide cuts data center power consumption in half

#69

Post by eaasen »

walrus01 wrote: post #13

They do have a good point here. If you do the total power budget on a typical 1U (discrete chassis, not blade) server which is packed full of a wall of 40mm fans pushing air, the highest speed screaming 40mm 12VDC fans can be 20W electrical load each . It's easy to "spend" at least 120W at maximum heat from the CPUs, in a dual socket system, just on the fans to pull air from the front/cold side of the server through…

FWIW, we had to have the idle speed of our fans lowered because the usual idle of around 5k RPM was WAY too much cooling. We generally run our fans at around 2.5kRPM (barely above idle). This is due to not only the larger fans, but also the fact that we optimized and prioritized as little restriction on airflow as possible. If you’ve taken apart a current gen 1U/2U server and then compare that to how little our airflow is restricted and how little our fans have to work, the 12X reduction becomes a bit clearer.

Re: How oxide cuts data center power consumption in half

#70

Post by throw0101d »

farawayea wrote: post #26

Their tech may be more than adequate today. Bigger businesses may not buy from a small startup company. They expect a lot more. Illumos is a less popular OS. It wouldn't be the first choice for the OS I'd rely on. Who writes the security mitigations for speculative execution bugs? Who patches CVEs in the shipped software which doesn't use Rust?

> Bigger businesses may not buy from a small startup company.

What would you classify Shopify as?

> One existing Oxide user is e-commerce giant Shopify, which indicates the growth potential for the systems available.

* https://blocksandfiles.com/2024/07/04/oxide-ships-first-clou...

Their CEO has tweeted about it:

* https://twitter.com/tobi/status/1793798092212367669

> Who writes the security mitigations for speculative execution bugs? Who patches CVEs in the shipped software which doesn't use Rust?

Oxide.

This is all a pre-canned solution: just use the API like you would an off-prem cloud. Do you worry about AWS patching stuff? And how many people purchasing 'traditional' servers from Dell/HPe/Lenovo worry about patching links like the LOM?

Further, all of Oxide's stuff is on Github, so you're in better shape for old stuff, whereas if the traditional server vendors EO(S)L something firmware-wise you have no recourse.

Post reply on HN