Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Helios: A distribution of Illumos powering the Oxide Rack

github.com

Post reply on HN
Print view View original

161–170 of 268 posts

Re: Helios: A distribution of Illumos powering the Oxide Rack

#161

Post by bewaretheirs »

refulgentis wrote: post #31
qmarchi wrote: post #27

Earlier quoted context omitted.

Small note, that's not true for Google Cloud, which runs on top of Linux, though modified. Disclaimer: Former Googler, Cloud Support

Another Xoogler here: any idea what they mean by it's not Linux at the bottom for other providers? Like, surely it's _some_ common OS? Either my binaries wouldn't run or AWS is reimplementing Linux so they can, which seems odd. Or are they just saying that the VM my binary runs on might be some predictable Linux version, but the underlying thing launching the VM could be anything?

When your programs are running on a VM, the linux that loads and runs your binaries is not at the bottom; that linux image runs inside a virtual machine which is constructed and supervised by a hypervisor which sits underneath it all. That hypervisor may run on the bare machine (or what passes for a bare machine what with all the sub-ring-zero crud out there), or may run on top of another OS which could be linux or something else. And even if there is linux in the middle and linux at the bottom they could be completely different versions of linux from releases made years apart.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#162

Post by antod »

refulgentis wrote: post #31
qmarchi wrote: post #27

Earlier quoted context omitted.

Small note, that's not true for Google Cloud, which runs on top of Linux, though modified. Disclaimer: Former Googler, Cloud Support

Another Xoogler here: any idea what they mean by it's not Linux at the bottom for other providers? Like, surely it's _some_ common OS? Either my binaries wouldn't run or AWS is reimplementing Linux so they can, which seems odd. Or are they just saying that the VM my binary runs on might be some predictable Linux version, but the underlying thing launching the VM could be anything?

> Or are they just saying that the VM my binary runs on might be some predictable Linux version, but the underlying thing launching the VM could be anything?

Yup. eg with Xen the hypervisor wasn't Linux, even if the privileged management VM (dom0) was Linux (or optionally NetBSD in the early days). The very small Xen hypervisor running on the bare metal was not a general purpose OS, and didn't expose any interface itself - it was well hidden and relied on dom0 for administration.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#163

Post by sarlalian »

GrumpySloth wrote: post #46
StillBored wrote: post #26

Earlier quoted context omitted.

Linux is a nightmare in the embedded/appliance space because one ends up just having platform engineers who spend their day fixing problems with the latest kernels, drivers, core libraries, etc, that the actual application depends on. Or one goes the route of 99% of the IoT/etc vendors, and never update the base OS and pray that there aren't any active exploits targeting it. This is why a lot of medium-sized companie…

CentOS wasn’t used in embedded systems.

Arista EOS is definitely CentOS Linux release 7.9.2009 (AltArch) based.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#164

Post by rjzzleep »

nwilkens wrote: post #118
EvanAnderson wrote: post #43

Earlier quoted context omitted.

I'm excited to see how this compares to SmartOS. I'm pretty heavily invested in SmartOS in my personal infrastructure but its future, post-Joyent acquisition, has been worrying me. I really wish I did work for an org big enough to use Oxide's gear. Not having to futz around with bogus IBM PC AT-type compatibility edifice, janky BMCs and iDRACs, hardware RAID controllers, etc, would be so unbelievably nice.

SmartOS is being actively developed since the aquisition from Joyent[1] in April 2022. We've released a new version every two weeks post acquisition, and are continuing to develop and invest. We also hold office hours events roughly every two weeks on Discord[2], and would love for you to stop by and ask any questions, or just listen along! [1]: https://www.tritondatacenter.com/blog/a-new-chapter-begins-f... [2]: htt…

IllumOS needs to attract new developers. To do that, the platform build needs to become a lot more straightforward. It's a pretty huge endeavour in my opinion. I'd be happy to help out on that regard, but in the past Joyent has not been very open to outside support.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#165

Post by throwawaaarrgh »

Jtsummers wrote: post #90
wmf wrote: post #70

Earlier quoted context omitted.

A handful of the team are more familiar with Illumos and the next hundred people they hire after that will be more familiar with Linux.

If your hiring decisions are always based on what people are currently familiar with, you'll always be stuck in the past. You may not even be able to use present day tooling and systems because they could be too new to hire people for. You're much better off hiring people who are capable of learning, and then giving them the opportunities to learn and advance their knowledge and skills.

Everyone is capable of learning. I can hire someone who is capable of learning Japanese. They can then try to teach the rest of the team Japanese. Does that mean it's a good idea to switch all our internal docs to Japanese? Maybe if I was building a startup in Japan. Similarly, writing internal docs in English for a startup in Japan would be of equal difficulty and value. Hooray, we're learning! And struggling more than needed to build a product.

You're better off hiring experienced people who are highly productive. If they're highly productive with one stack, it makes no sense to change their stack so they're no longer productive, or hiring people who aren't familiar with it and waiting for them to become productive.

There's nothing wrong with using old, well established things. They're quite often better than new things. As long as they're still supported, just use whatever builds a working product. It's the end product that matters.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#166

Post by steveklabnik »

mihaic wrote: post #149

I'm really curious: what kind of workload would companies want to run on a custom Unix that isn't Linux/Mac/BSD? I'm rooting for more mature OS diversity, I just have no idea who the end users would be and what their needs would look like.

This is not a user-facing detail of the product. Customers run VMs on the rack, they do not build their applications for illumos. They're gonna run whatever operating system in those VMs that they need to accomplish their goals.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#167

Post by steveklabnik »

ahmedfromtunis wrote: post #147

It is great that the software is open-source, but would be you useful to be deployed on other hardware? And what would happen if, for whatever reason, a company can no longer purchase Oxide racks, will it need to start over its infra, or can it just build around Oxide hardware?

It is not likely that it would be immediately useful outside of our hardware, but the main thing they're doing is deploying virtual machines. If they decided to no longer use the Oxide rack they have purchased, they would move their VMs to whatever infrastructure they choose to succeed it.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#168

Post by rmccue »

technofiend wrote: post #148
busterarm wrote: post #45

Not that I'm not rooting for Oxide, but their product is still so niche and early stage that I can't imagine any actual businesses buying their stuff for a long time. They only just shipped their first rack to their first customer at the end of last summer and it's Idaho National Laboratory. State research institutions are basically the only entities positioned to gamble on this right now.

It is somewhat niche, but Broadcom's purchase of VMWare now puts 0xide closer to Nutanix in that you can go buy a fully supported virtualization platform from a vendor who welcomes your business. I don't know the actual number, but it seems Broadcom is only interested in enterprise customers with huge annual spends.

With Broadcom’s plan for VMWare, Oxide certainly seems to have had excellent timing here.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#170

Post by kaliszad »

magnawave wrote: post #116
throwup238 wrote: post #33

Earlier quoted context omitted.

The best elevator pitch I've heard is "AWS APIs for on-prem datacenters". They make turn-key managed racks that behave just like a commercial cloud would with all the APIs for VM, storage, and network provisioning and integration you'd expect from AWS, except made to deploy in your company's datacenter under your control.

I guess the wildcard is price. AWS's pricing model works kinda at their OMG eyewatering scale - aka all the custom hardware they design is highly cost optimized, but just doing custom hardware has a notable cost. This is easily covered by their scale, to make for their famous margins. [during their low scale times, they did use a good bit of HP/Dell, etc] Oxide seems to be no different (super custom hardware) only ma…

Oxide seems to be a lot more efficient than a rack full of 1U servers with each having 2 PSUs + 2 ToR switches + 1 management switch somewhere for all the OOBMs. All those little fans and power conversions eat a lot of power, the fans and the PSUs all cost something too. Also, have fun managing all of that in a secure manner or debugging anything at all. Once you add the VMware licensing you might end up with more or less the same cost up front and quite likely higher overall cost. And I am not even beginning to talk about racking/ stacking of the whole rack. I haven't seen much support even when Dell/EMC owned VMware and together produced the VXRail lineup and the company I used to work for was presented as the reference project in Saxony, Germany at that time. All of the boxes would add up to about 2 standard racks but it was representative of the other bigish customers in that area and time.

I imagine, some of the customers will order 1-2 racks half full and over a few years possibly add a few sleds, these will probably demand great GUI/ manual experience and possibly competitive Oracle/ SAP/ MSSQL benchmarks and I can imagine Veeam integreation. Other customers such as the DoE or some big enterprise customers will order whole rows of racks and demand perfect automation options. That is just a guess.

Post reply on HN