Helios: A distribution of Illumos powering the Oxide Rack • Hacker News Board

Re: Helios: A distribution of Illumos powering the Oxide Rack

#231

Post by antranigv » Tue, Jan 30, 2024, 10:39 AM UTC

yjftsjthsd-h wrote: post #193

Sweet:) And a big thanks for writing what appears to be clear and straightforward documentation; IMO that's an area that the illumos community has historically struggled with. And seeing a new source release talking about consolidations gives me the warm fuzzies, even if this does seem to depart from the traditional gate paradigm unless I'm seriously misreading the repo organization here. Some (mostly tooling) questi…

> is that a defect in upstream, or is the local sh not completely posix compatible? AFAIK it's an issue with upstream. Just like most open-source projects, there is Linuxism/Bashism in there.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#232

Post by antranigv » Tue, Jan 30, 2024, 10:47 AM UTC

tw04 wrote: post #201

steveklabnik wrote: post #166

Earlier quoted context omitted.

This is not a user-facing detail of the product. Customers run VMs on the rack, they do not build their applications for illumos. They're gonna run whatever operating system in those VMs that they need to accomplish their goals.

That being said - there's something to be said for enterprise support. Are there plans to support importing/converting/running third party OVAs? Many vendors will support something running in KVM, I can't recall the last time I saw b-hyve as a supported hypervisor. I'd imagine as broadcom slowly destroys vmware's market share vendors will look to alternatives, but I doubt b-hyve is even a blip on their radar at this…

OVAs are basically ZIP files with some XML. If you want, you can convert an OVA to RAW image or VMDK or whatever the latest fancy format is, and bhyve can boot that for you. Better to use Raw.

bhyve, unlike other "famous" hypervisors is pretty stable, has good enough virtualized drivers (altho I'm sure Oxide has made it better) and can boot a VM with 1.5 TB of RAM and 240 vCPU[1]. Something I was not able to do with anything other than bhyve.

I know this is HackerNews, so I have to say it, marketing != engineering. Just because the FreeBSD project's marketing suck, doesn't mean engineering is bad. usually it better than the mainstream ones.

1: https://antranigv.am/posts/2023/10/bhyve-cpu-allocation-256/

Re: Helios: A distribution of Illumos powering the Oxide Rack

#233

Post by rbanffy » Tue, Jan 30, 2024, 1:43 PM UTC

unethical_ban wrote: post #222

icybox wrote: post #143

Earlier quoted context omitted.

I've been running smartos at least since 2015 where I co-located my server. There have been times where I felt like giving up, but people like danmcd, jperkin and others always stepped in and fixed what needed to be fixed for LX to be usable and working. (Keeping java updated and running is hard, uphill battle. Thanks!) I always ran a mixture of OS and LX zones and bcantrill's t-shirt with "Save the whales, kill your…

And here I am, having compared the SmartOS documentation and ease of installation to Proxmox... and with very few complaints am using Proxmox to host a file server on bare metal/Samba container and a OPNSense on VM. I remember buying the OpenSolaris Bible in 2008, getting really excited to dig into my second Unix (after FreeBSD). And then, the Sun went down on me... and I stuck with Ubuntu 10 years.

> and I stuck with Ubuntu 10 years.

For a while Nexenta had an Ubuntu running on the OpenSolaris kernel.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#234

Post by rbanffy » Tue, Jan 30, 2024, 1:44 PM UTC

yjftsjthsd-h wrote: post #209

DominoTree wrote: post #207

Earlier quoted context omitted.

>> Oxide is pretty much the company I dream to work at, both for the tech stack, plus the people working there. Thought I was the only one :P

I mean, I phrase it as "my dream job is systems integration at Sun" (but Oxide is the living equivalent)

Sun had some lovely desktop hardware. They also had SPARC.

I really miss Sun.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#235

Post by mbreese » Tue, Jan 30, 2024, 1:53 PM UTC

amluto wrote: post #204

mbreese wrote: post #196

Earlier quoted context omitted.

> This isn’t to be polite or to make money — it’s to foster adoption among developers, which drives sales. I get that in theory… and it makes sense most of the time. I’m not sure it does this time though. You don’t exactly “target” Oxide as an OS or platform. Rather, you use it to run VMs on. Those VMs are whatever you want. Other than that, I’m not sure what else having a home-lab version of Oxide would look like. A…

People target AWS and GCP and Azure, and they write actual code that interacts with them, do test deployments there, and do real deployments there.

Right, but you need to be at a particularly high scale when you need to write code that directly interacts with a cloud provider. I can think of no use case where you are at that level that can also scale down to a home lab setup.

Yes, you can do slimmed down cloud deployments, but you're still not running the (actual) S3 or EC2 backends at home.

At most, you’re talking a K8s based deployment (for a workload that could scale up or down). But that’s also not at the level of working with Oxide directly. And I doubt Oxide wants to get into the business of selling access to their own public cloud.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#236

Post by throwawaaarrgh » Tue, Jan 30, 2024, 1:55 PM UTC

Jtsummers wrote: post #180

throwawaaarrgh wrote: post #165

Earlier quoted context omitted.

Everyone is capable of learning. I can hire someone who is capable of learning Japanese. They can then try to teach the rest of the team Japanese. Does that mean it's a good idea to switch all our internal docs to Japanese? Maybe if I was building a startup in Japan. Similarly, writing internal docs in English for a startup in Japan would be of equal difficulty and value. Hooray, we're learning! And struggling more t…

> Everyone is capable of learning. I can hire someone who is capable of learning Japanese. They can then try to teach the rest of the team Japanese. Does that mean it's a good idea to switch all our internal docs to Japanese? The difference between Japanese and English is much, much bigger than the difference between one Unix OS and one Unix-like OS. This is a remarkably disingenuous argument. If you really don't und…

You're welcome to see it that way if you want. But if you think you can get to know a completely new kernel, OS, etc in a short amount of time, backwards and forwards, you're equally as disingenuous. You can get by editing a few lines in a pinch, but you could equally just learn a few Japanese phrases. Proper understanding requires a deep knowledge that comes from practice and experience with subtle complexity and context.

(Japanese isn't so radically different from English, it mostly just has more words for more contexts. In many ways it's simpler than English. It would be harder to go from Java to Haskell, with their many different language paradigms)

Re: Helios: A distribution of Illumos powering the Oxide Rack

#237

Post by tw04 » Tue, Jan 30, 2024, 4:50 PM UTC

antranigv wrote: post #232

tw04 wrote: post #201

Earlier quoted context omitted.

That being said - there's something to be said for enterprise support. Are there plans to support importing/converting/running third party OVAs? Many vendors will support something running in KVM, I can't recall the last time I saw b-hyve as a supported hypervisor. I'd imagine as broadcom slowly destroys vmware's market share vendors will look to alternatives, but I doubt b-hyve is even a blip on their radar at this…

OVAs are basically ZIP files with some XML. If you want, you can convert an OVA to RAW image or VMDK or whatever the latest fancy format is, and bhyve can boot that for you. Better to use Raw. bhyve, unlike other "famous" hypervisors is pretty stable, has good enough virtualized drivers (altho I'm sure Oxide has made it better) and can boot a VM with 1.5 TB of RAM and 240 vCPU[1]. Something I was not able to do with…

There's two parts to the question. There's the file itself and the underlying hardware. The wiki is pretty light on details, does it actually support emulating the same hardware as vmware? I'd assume no to the vmxnet devices but Intel E1000? Adaptec SCSI adapters? Similar USB and VGA?

A lot of the vendor-provided OVAs cut out a bunch of hardware support with the assumption that they only need to support vmware emulated hardware.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#238

Post by segmondy » Tue, Jan 30, 2024, 6:11 PM UTC

I really hope Oxide succeeds. I thought they were crazy when they announced what they were going to do. It's not the kind of play you see from a start up, but they were determined. Most folks that started the race at that time are out. I hope their computer is ready to deploy GPUs. Deploying multiple GPUs today is a freaking pain.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#239

Post by sunshowers » Tue, Jan 30, 2024, 7:15 PM UTC

bbkane wrote: post #224

sunshowers wrote: post #50

Earlier quoted context omitted.

(I work at Oxide.) Bryan is just one out of several illumos experts here. If none of those were around, sure, maybe we wouldn't have picked illumos -- but then we'd be unrecognizably different. I came into Oxide with a Linux background and zero knowledge of illumos. Learning about DTrace especially has been great.

I'd like to learn DTrace (especially after the recent 20yr podcast episode), but I worry it'll never make into mainstream Linux debugging, and hence only useful for more niche jobs.

Your concern is completely reasonable -- a thing I'd add though is that both Windows and macOS have DTrace support.

Re: Helios: A distribution of Illumos powering the Oxide Rack

#240

Post by bbkane » Tue, Jan 30, 2024, 7:49 PM UTC

sunshowers wrote: post #239

bbkane wrote: post #224

Earlier quoted context omitted.

I'd like to learn DTrace (especially after the recent 20yr podcast episode), but I worry it'll never make into mainstream Linux debugging, and hence only useful for more niche jobs.

Your concern is completely reasonable -- a thing I'd add though is that both Windows and macOS have DTrace support.

I was excited, but it looks like both MacOS and Windows require special admin permissions for my laptop that I doubt my work would approve (completely reasonable to require this, it just makes it unusable for me).