Live data from Hacker News

Servers as they should be – shipping early 2022

oxide.computer

61–70 of 264 posts

Re: Servers as they should be – shipping early 2022

#61
post #41
post #37

Earlier quoted context omitted.

"Attest" refers to https://en.wikipedia.org/wiki/Trusted_Computing#REMOTE-ATTES... , not forced code signing

I'm familiar with the concept. Does this mean that attestation to a different root of trust than Oxide will also be feasible, and that this is just a default?

It's an "enterprise" product, so you can be certain that some amount of extra money will buy you that capability (if it's not already included).

Re: Servers as they should be – shipping early 2022

#62
post #57

It sure looks pretty, but appears to be - - dedicated to virtualization, done their way - rather inflexible in hardware specs - vendor-locked at the rack - if you have hardware from someone else, it can't live in the same cabinet I guess if you just want a pretty data center in a box and look like what they consider a 'normal' enterprise to be, it might appeal. But I'm not sure how many people asked for Apple-style h…

A lot of customers are asking for private cloud.

Re: Servers as they should be – shipping early 2022

#64
post #21

First, congrats! But second, I'd love to understand the compute vs storage tradeoff chosen here. Looking at the (pretty!) picture [1], I was shocked to see "Wow, it's mostly storage?". Is that from going all flash? Heading to https://oxide.computer/product for more details, lists: - 2048 cores - 30 TB of memory - 1024 TB of flash (1 PiB) Given how much of the rack is storage, I'm not sure which Milan was chosen (and…

+1 to congrats -- my read on this:

- There's a bunch of RJ45 up top that I don't quite understand :)

- A bunch of storage sleds

- A compute sled, 100G QSFP switch, compute sled sandwich

- Power distribution (rectifiers, I'd think, unless it's AC to the trays?)

- Another CSC sandwich

- More storage.

I assume in reality we'd have many more cables making things less pretty, given the number of front-facing QSFPs on those ToRs.

Re: Servers as they should be – shipping early 2022

#65
post #57

It sure looks pretty, but appears to be - - dedicated to virtualization, done their way - rather inflexible in hardware specs - vendor-locked at the rack - if you have hardware from someone else, it can't live in the same cabinet I guess if you just want a pretty data center in a box and look like what they consider a 'normal' enterprise to be, it might appeal. But I'm not sure how many people asked for Apple-style h…

> - dedicated to virtualization, done their way

> - rather inflexible in hardware specs

> - vendor-locked at the rack - if you have hardware from someone else, it can't live in the same cabinet

This describes legacy IBM platforms quite well. If they can leverage hyperscaling tech to be better and cheaper than what IBM is currently offering, that's enough to make it worthwhile.

Re: Servers as they should be – shipping early 2022

#66
post #41
post #37

Earlier quoted context omitted.

"Attest" refers to https://en.wikipedia.org/wiki/Trusted_Computing#REMOTE-ATTES... , not forced code signing

I'm familiar with the concept. Does this mean that attestation to a different root of trust than Oxide will also be feasible, and that this is just a default?

Oxide makes the hardware, so it makes sense to use them as the root of trust since you already have to trust them to not make backdoored hardware. Why bother adding more parties? Also, for remote attestation to make sense, it needs to be done in the hardware itself (ie. keys burned into the silicon). I'm not sure how that's supposed to work if you add your own keys, or whether that would even make sense.

Re: Servers as they should be – shipping early 2022

#67
post #33

Earlier quoted context omitted.

No, single system image is really expensive (see Superdome Flex) and most workloads can't justify that cost.

Could one reimplement SSI at the OS layer, similar to existing distributed OS's? Distributed shared memory is usually dismissed because of the overhead involved in typical scenarios, but this kind of hardware platform might make it feasible.

There was Mosix at the OS layer in the 1990s and Virtual Iron at the hypervisor layer in the aughts. I think the cost and performance of software SSI just doesn't intersect with demand anywhere.

Re: Servers as they should be – shipping early 2022

#68
post #57

It sure looks pretty, but appears to be - - dedicated to virtualization, done their way - rather inflexible in hardware specs - vendor-locked at the rack - if you have hardware from someone else, it can't live in the same cabinet I guess if you just want a pretty data center in a box and look like what they consider a 'normal' enterprise to be, it might appeal. But I'm not sure how many people asked for Apple-style h…

Why is it important what kind of virtualization? It works and since it is built for this hardware it will likely be more reliable then anything you're putting together yourself.

The specs are damn good. When it is all top-of-the-line, inflexibility is kind of a mute point. Where else are you going to go?

> But I'm not sure how many people asked for Apple-style hardware in the DC.

Well integrated, performant and reliable hardware that runs VMs where you can put anything on it is pretty much all everyone running their own hardware is looking for.

Honestly I am surprised how many here completely misunderstand what their value proposition is.

Re: Servers as they should be – shipping early 2022

#69

Congrats Oxide team! More competition in this space is always a good thing. I'm curious about management. Can the rack operate completely standalone? I assume when you have multiple there will be some management abstraction above the rack layer? The closest direct equivalent that I can think of to this is AWS outposts. Are there any others that I'm forgetting?

The density they're getting here is significantly higher than AWS Outposts, which is interesting. The top-end (~$600k) AWS Outposts seem to max out at around 1k CPUs and 4.5 TB RAM in a rack (e.g. 12x m5.24xlarge = 12x 384 GB), while this rack can house 2k CPUs and 30 TB (!) RAM.

Yes, well it isn't that dense either. As I have written, it's 32 CPUs (16x 2 CPUs). 1 TB of RAM per CPU is not that huge a deal, it's perhaps 16x 64 GB (Milan uses 8 channels, 2 DIMMs per channel is reasonable), if you consider that is 16 GB of RAM per core. In HPC, you would probably shrink it to 1/4 of the volume (half width, 1 U dual socket server). Oxide probably focuses on optimal thermal efficiency since their limit isn't the space so much as the power density/ max. power per rack in existing DCs, which they are already pushing hard. (Of course they have lower power options too but they probably will not use 2048 cores.)

Re: Servers as they should be – shipping early 2022

#70
post #64
post #21

First, congrats! But second, I'd love to understand the compute vs storage tradeoff chosen here. Looking at the (pretty!) picture [1], I was shocked to see "Wow, it's mostly storage?". Is that from going all flash? Heading to https://oxide.computer/product for more details, lists: - 2048 cores - 30 TB of memory - 1024 TB of flash (1 PiB) Given how much of the rack is storage, I'm not sure which Milan was chosen (and…

+1 to congrats -- my read on this: - There's a bunch of RJ45 up top that I don't quite understand :) - A bunch of storage sleds - A compute sled, 100G QSFP switch, compute sled sandwich - Power distribution (rectifiers, I'd think, unless it's AC to the trays?) - Another CSC sandwich - More storage. I assume in reality we'd have many more cables making things less pretty, given the number of front-facing QSFPs on thos…

They use a bus bar design. That is what @bcantrill also said in an interview.
Post reply on HN