Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Helios: A distribution of Illumos powering the Oxide Rack

github.com

Post reply on HN
Print view View original

11–20 of 268 posts

Re: Helios: A distribution of Illumos powering the Oxide Rack

#11

Post by tonyarkles »

vvern wrote: post #4
sneak wrote: post #2

I know they’re ex-Sun, but is there any real technical benefit for choosing not-Linux (for their business value prop)? I know of the technical benefits of illumos over linux, but does that actually matter to the customers who are buying these? Aren’t they opening a whole can of worms for ideology/tradition that won’t sell any more computers? As someone who runs Linux container workloads, the fact that this is fundame…

> yes I know it runs Linux binaries unmodified Is it that it runs Linux binaries unmodified or that it runs vms and manages VMs which run Linux, and as an end-user, that's what you run your software in?

As far as I recall it's not a VM. They run in "LX Branded Zones" which does require a Linux userland so that the binaries can find their libraries etc but Zones are more like "better cgroups than cgroups, a decade earlier" than VMs.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#12

Post by bcantrill »

vvern wrote: post #4
sneak wrote: post #2

I know they’re ex-Sun, but is there any real technical benefit for choosing not-Linux (for their business value prop)? I know of the technical benefits of illumos over linux, but does that actually matter to the customers who are buying these? Aren’t they opening a whole can of worms for ideology/tradition that won’t sell any more computers? As someone who runs Linux container workloads, the fact that this is fundame…

> yes I know it runs Linux binaries unmodified Is it that it runs Linux binaries unmodified or that it runs vms and manages VMs which run Linux, and as an end-user, that's what you run your software in?

It runs VMs -- so it doesn't just run Linux binaries unmodified, it runs Linux kernels unmodified (and, for that matter, Windows, FreeBSD, OpenBSD, etc.).

Re: Helios: A distribution of Illumos powering the Oxide Rack

#13

Post by jeffbee »

sneak wrote: post #2

I know they’re ex-Sun, but is there any real technical benefit for choosing not-Linux (for their business value prop)? I know of the technical benefits of illumos over linux, but does that actually matter to the customers who are buying these? Aren’t they opening a whole can of worms for ideology/tradition that won’t sell any more computers? As someone who runs Linux container workloads, the fact that this is fundame…

Do you have the same gut reaction to ESXi?

Re: Helios: A distribution of Illumos powering the Oxide Rack

#14

Post by steveklabnik »

sneak wrote: post #2

I know they’re ex-Sun, but is there any real technical benefit for choosing not-Linux (for their business value prop)? I know of the technical benefits of illumos over linux, but does that actually matter to the customers who are buying these? Aren’t they opening a whole can of worms for ideology/tradition that won’t sell any more computers? As someone who runs Linux container workloads, the fact that this is fundame…

> does that actually matter to the customers who are buying these?

It's not like we specifically say "oh btw there's illumos inside and that's why you should buy the rack." It's not a customer-facing detail of the product. I'm sure most will never even know that this is the case.

What customers do care about is that the rack is efficient, reliable, suits their needs, etc. Choosing illumos instead of Linux here is a choice made to help effectively deliver on that value. This does not mean that you couldn't build a similar product on top of Linux inherently, by the way, just that we decided illumos was more fit for purpose.

This decision was made with the team, in the form of an RFD[1]. It's #26, though it is not currently public. The two choices that were seriously considered were KVM on Linux, and bhyve on illumos. It is pretty long. In the end, a path must be chosen, and we chose our path. I do not work on this part of the product, but I haven't seen any reason to believe it has been a hindrance, and probably is actually the right call.

> the fact that this is fundamentally not-Linux (yes I know it runs Linux binaries unmodified) would be a reason against buying it, not for.

I am curious why, if you feel like elaborating. EDIT: oh just saw your comment down here: https://news.ycombinator.com/item?id=39180814

1: https://rfd.shared.oxide.computer/

Re: Helios: A distribution of Illumos powering the Oxide Rack

#15

Post by thinkingkong »

sneak wrote: post #2

I know they’re ex-Sun, but is there any real technical benefit for choosing not-Linux (for their business value prop)? I know of the technical benefits of illumos over linux, but does that actually matter to the customers who are buying these? Aren’t they opening a whole can of worms for ideology/tradition that won’t sell any more computers? As someone who runs Linux container workloads, the fact that this is fundame…

This has been / will be the market education challenge; Its the same one Joyent had with SmartOS. Theyre correctly pointing out that the end user or operator will basically never interact with this layer, but it does cause some knee-jerk reactions. All that said, there are some pretty great technical benefits to using illumos derived systems the least of which is the teams familiarity and ability to do real diagnosis on production issues. I wont put words in anyones mouth but I suspect thats going to be critical for them as they support customer deployments w/o direct physical access.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#16

Post by bcantrill »

tonyarkles wrote: post #11
vvern wrote: post #4

Earlier quoted context omitted.

> yes I know it runs Linux binaries unmodified Is it that it runs Linux binaries unmodified or that it runs vms and manages VMs which run Linux, and as an end-user, that's what you run your software in?

As far as I recall it's not a VM. They run in "LX Branded Zones" which does require a Linux userland so that the binaries can find their libraries etc but Zones are more like "better cgroups than cgroups, a decade earlier" than VMs.

No, it's a VM, running a bhyve-based hypervisor, Propolis.[0] LX branded zones were/are great -- but for absolute fidelity one really needs VMs.

[0] https://github.com/oxidecomputer/propolis

Re: Helios: A distribution of Illumos powering the Oxide Rack

#17

Post by steveklabnik »

quux wrote: post #10
sneak wrote: post #2

I know they’re ex-Sun, but is there any real technical benefit for choosing not-Linux (for their business value prop)? I know of the technical benefits of illumos over linux, but does that actually matter to the customers who are buying these? Aren’t they opening a whole can of worms for ideology/tradition that won’t sell any more computers? As someone who runs Linux container workloads, the fact that this is fundame…

Aren't they also ex-Joyent? Joyent ran customer VMs in prod on Illumos for many years so there's a lot of experience there.

Many people, including part of the founding team, are ex-Joyent, yes. Some also worked at Sun, on the operating systems that illumos is ultimately derived from.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#18

Post by littlestymaar »

quux wrote: post #10
sneak wrote: post #2

I know they’re ex-Sun, but is there any real technical benefit for choosing not-Linux (for their business value prop)? I know of the technical benefits of illumos over linux, but does that actually matter to the customers who are buying these? Aren’t they opening a whole can of worms for ideology/tradition that won’t sell any more computers? As someone who runs Linux container workloads, the fact that this is fundame…

Aren't they also ex-Joyent? Joyent ran customer VMs in prod on Illumos for many years so there's a lot of experience there.

bcantrill used to work at Sun then became CTO at Joyent, so the reason why Joyent ran Illumos is probably the same reason as why Oxide is, because Cantrill likes it and judges that it's a good fit for what they are doing.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#19

Post by steveklabnik »

littlestymaar wrote: post #18
quux wrote: post #10

Earlier quoted context omitted.

Aren't they also ex-Joyent? Joyent ran customer VMs in prod on Illumos for many years so there's a lot of experience there.

bcantrill used to work at Sun then became CTO at Joyent, so the reason why Joyent ran Illumos is probably the same reason as why Oxide is, because Cantrill likes it and judges that it's a good fit for what they are doing.

As I elaborated above, bcantrill did not decree that we must use illumos. Technical decisions are not handed down from above at Oxide.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#20

Post by stonogo »

jeffbee wrote: post #13
sneak wrote: post #2

I know they’re ex-Sun, but is there any real technical benefit for choosing not-Linux (for their business value prop)? I know of the technical benefits of illumos over linux, but does that actually matter to the customers who are buying these? Aren’t they opening a whole can of worms for ideology/tradition that won’t sell any more computers? As someone who runs Linux container workloads, the fact that this is fundame…

Do you have the same gut reaction to ESXi?

I sure do. We've finally got to a place where we don't need weird hardware tricks to containerize workloads -- this is why a lot of shops pursue docker-like ops for production. When I buy hardware, long-term maintenance is a factor, and when my whole operations fleet relies on ESX, or in this case a Solaris fork, I'm now beholden to one company for support at that layer. Buying a rack of Supermicro gear and running RHEL or SLES with containerized orchestration on top means I can, in a pinch, hire experts anywhere to work on my systems.

I have no reason to believe Oxide would be anything but responsive and effective in supporting their systems, but introducing bespoke software this deep in the stack severely curtails my options if things get bad.

Post reply on HN