I'm really curious: what kind of workload would companies want to run on a custom Unix that isn't Linux/Mac/BSD? I'm rooting for more mature OS diversity, I just have no idea who the end users would be and what their needs would look like.
Helios: A distribution of Illumos powering the Oxide Rack
221–230 of 268 posts
Re: Helios: A distribution of Illumos powering the Oxide Rack
#222Earlier quoted context omitted.
I had been using SmartOS for a long time but finally had to bite the bullet and give up. I ended up deciding on Proxmox on a ZFS root and am quite happy with it.
I've been running smartos at least since 2015 where I co-located my server. There have been times where I felt like giving up, but people like danmcd, jperkin and others always stepped in and fixed what needed to be fixed for LX to be usable and working. (Keeping java updated and running is hard, uphill battle. Thanks!) I always ran a mixture of OS and LX zones and bcantrill's t-shirt with "Save the whales, kill your…
I remember buying the OpenSolaris Bible in 2008, getting really excited to dig into my second Unix (after FreeBSD). And then, the Sun went down on me... and I stuck with Ubuntu 10 years.
Re: Helios: A distribution of Illumos powering the Oxide Rack
#223I know they’re ex-Sun, but is there any real technical benefit for choosing not-Linux (for their business value prop)? I know of the technical benefits of illumos over linux, but does that actually matter to the customers who are buying these? Aren’t they opening a whole can of worms for ideology/tradition that won’t sell any more computers? As someone who runs Linux container workloads, the fact that this is fundame…
> does that actually matter to the customers who are buying these? It's not like we specifically say "oh btw there's illumos inside and that's why you should buy the rack." It's not a customer-facing detail of the product. I'm sure most will never even know that this is the case. What customers do care about is that the rack is efficient, reliable, suits their needs, etc. Choosing illumos instead of Linux here is a c…
Re: Helios: A distribution of Illumos powering the Oxide Rack
#224Earlier quoted context omitted.
I saw your comment[1] after I wrote mine, but I'm not saying that he's forcing you guys to use it (that would not a good way of being a CTO at a start-up…), but that doesn't prevent him from advocating for solutions he believes in. Would you say that Oxide would have chosen Illumos if he wasn't part of the company? [1]: https://news.ycombinator.com/item?id=39180706
(I work at Oxide.) Bryan is just one out of several illumos experts here. If none of those were around, sure, maybe we wouldn't have picked illumos -- but then we'd be unrecognizably different. I came into Oxide with a Linux background and zero knowledge of illumos. Learning about DTrace especially has been great.
Re: Helios: A distribution of Illumos powering the Oxide Rack
#225Earlier quoted context omitted.
Can you get me excited? I spent 20 seconds browsing the homepage and walked away with "so the idea is vertical integration for on-premise server purchases? On custom OS? Why? Why would people pay a premium?" But immediately got myself to "what does a server OS do anyway, doesn't it just launch VMs? You don't need Linux, just the ability to launch Linux VMs" Tell me more? :)
Having a solid on-prem rack product to me is a great thing. I like IaaS services a lot, don't get me wrong, and I think they're the right pick for a bunch of cases, but on-prem servers also have their "place in the sun", so to speak :) I could present any number of justifications that I don't think I'm qualified enough to defend, but the gist is that at the bare minimum, I'm glad the option exists. As to why I'm pers…
Of all of the 6 infrastructures I've seen, only 1 of them is half decent, with 6 dedicated teams around the datacenter working closely together (by dedicated I mean, nothing is required of them concerning the core software product that the company develops). Network, Unix/Virtu, Windows, Storage, PC, and datacenter. That's 30+ people just to run a couple big datacenters and a few more server rooms. The service was actually quite good with VMs/zones delivered under an hour and most tech issues solved in half a day. The other infrastructures were either bigger or smaller, with more or less people, and were all terrible, sometimes needing weeks of email exchanges with excel attached to get a single VM.
AWS was the dream everywhere I went for everybody. Oxide may be coming out with a product that will solve a LOT of issues. SmartOS/IllumOS has all the tech to be self-sufficient (virtualization, storage, SDN...), add support for networking and storage and you get a complete product that a handful of people can run (well, you still need a windows team in most cases but fine).
Re: Helios: A distribution of Illumos powering the Oxide Rack
#226Not that I'm not rooting for Oxide, but their product is still so niche and early stage that I can't imagine any actual businesses buying their stuff for a long time. They only just shipped their first rack to their first customer at the end of last summer and it's Idaho National Laboratory. State research institutions are basically the only entities positioned to gamble on this right now.
I work at a recently IPO'd tech company. Oxide was a strong consideration for us when evaluating on prem. The pitch lands among folks who still think "on prem.... ew". Looks like a cloud like experience on your own hardware. If only it were as cheap as dell...
That goes double if it's your CTO...which is exactly what ended up happening with us.
I'm not saying "no, never", but clearly "no, not right now".
Re: Helios: A distribution of Illumos powering the Oxide Rack
#227Earlier quoted context omitted.
Where, exactly, are you getting these 3FTEs qualified to touch production OpenStack infra, for more than a year, where their aggregate cost is less than a rack of equipment?
if you need OpenStack you're not running one rack, but a couple dozen.
Re: Helios: A distribution of Illumos powering the Oxide Rack
#228Earlier quoted context omitted.
SmartOS is being actively developed since the aquisition from Joyent[1] in April 2022. We've released a new version every two weeks post acquisition, and are continuing to develop and invest. We also hold office hours events roughly every two weeks on Discord[2], and would love for you to stop by and ask any questions, or just listen along! [1]: https://www.tritondatacenter.com/blog/a-new-chapter-begins-f... [2]: htt…
IllumOS needs to attract new developers. To do that, the platform build needs to become a lot more straightforward. It's a pretty huge endeavour in my opinion. I'd be happy to help out on that regard, but in the past Joyent has not been very open to outside support.
I also think you're overestimating the extent to which make(1S) is the reason we're not more popular than Linux. There are any number of more relevant factors that make someone choose one operating system or another. Also, certainly for me personally my goal is not world domination, merely the sustainable maintenance of a body of software that helps me solve the problems that I work on, and which I enjoy using and developing as a result.
I agree we need (as do all projects!) new developers, both now, and over the long term. We work as we can to make improvements to the build process, and the documentation. We are a relatively niche project, but we do attract new developers from time to time, and we're making changes at least as rapidly as we ever have in the past. There are a number of actively maintained illumos distributions (OmniOS, SmartOS, Tribblix, OpenIndiana, and now Helios) and there are a variety of commercial interests that ship more proprietary appliances on top of an illumos base. For our part at Oxide we continue to encourage our staff to get involved with illumos development as it makes sense for them, and we try to offer resources and assistance to the broader community as well.
If you would like to contribute, we have a guide to getting started: https://illumos.org/docs/contributing/
Please, though, it's "illumos", not "IllumOS"!
Re: Helios: A distribution of Illumos powering the Oxide Rack
#229Earlier quoted context omitted.
IllumOS needs to attract new developers. To do that, the platform build needs to become a lot more straightforward. It's a pretty huge endeavour in my opinion. I'd be happy to help out on that regard, but in the past Joyent has not been very open to outside support.
I've noticed you make this comment repeatedly when illumos is mentioned on HN. I think you're underestimating the irreducible complexity of the build process for what is essentially a whole UNIX operating system, save for a few external dependencies. It's not just a kernel, but an extensive set of user mode libraries and executables. The build is complex in part because it's a complex body of software. I also think y…
Even in the early 2000s linux had a make menuconfig or make xconfig setting to build linux. And yes this is different, it's a posix distribution. Yocto was a relatively niche project as well and it also addresses the issue of building a collecting of posix applications into a big project, so does gentoo's stage.
I'm sure that at the time of it's creation OpenSolaris was ahead of its curve, but that's how many years ago? You know as well as I do that sprinkling LD_LIBRARY_PATHs here and there and then removing undocumented dot files here and there isn't really a sane way to handle such a build process for a curious third party. Most will probably drop it before it gets to that point.
There have been many many projects that have reworked their entire build architecture, some of which took years to flesh out fully.
What needs to happen for illumos to get a boost of development in the long term is:
1. first for you to acknowledge on a political level that there is an issue that needs to be addressed here, and
2. to then work with the community, and it doesn't have to be across the board, but you need to be willing to invest in some experts and some people interested in solving this, so they can grind out something that is more sane in this current world.
"Read our getting started guide" isn't really all that useful, when most of the complex issues happen after that and are often met with "this isn't how we do things".
Re: Helios: A distribution of Illumos powering the Oxide Rack
#230Earlier quoted context omitted.
I've noticed you make this comment repeatedly when illumos is mentioned on HN. I think you're underestimating the irreducible complexity of the build process for what is essentially a whole UNIX operating system, save for a few external dependencies. It's not just a kernel, but an extensive set of user mode libraries and executables. The build is complex in part because it's a complex body of software. I also think y…
I do, yes, but your comment makes it clear that this is a problem that you either don't really think of as a problem, or that you don't know how to address. Building open source communities is hard work. Telling everyone how amazing your product is(even if it is), is only a small part of it. The lesson to take away from your time at Joyent should be that, that way of community building didn't work, and there needs to…
I obviously don't have any context about the issues you were facing at the time, and I can't really figure it out based on the advice you ostensibly received. I'm definitely sorry if we have lead you astray in the past, but those are not workarounds I would encourage people to use today. If there's some aspect of the build process that requires workarounds like you're describing, it's definitely a bug and we'll fix it when we're made aware as best we can.
As for the rest of it, I think you're putting the cart before the horse on some level. An operating system is a large and complex thing to work on, regardless of whether it's built with make or ninja or bazel or whatever other build tool.
The Rust toolchain is another similarly complex body of software, which also has a large and at times inscrutable build process. I know because I have personally contributed to it, and had to figure out how to get it to work. Rust obviously has more active contributors than illumos, but it also has vastly more active _users_ -- it is a body of software that has broad applicability to many people and the work they do.
For illumos to continue to succeed as an actively maintained project, what we need to do is continue to inspire _users_ to want to use it. Nobody wants to work on an operating system they don't personally need to use at all. We draw contributions today from a mixture of community driven distributions making fixes or adding features, and by people employed by companies like Oxide who have a vested economic interest in the deployment of the software.
None of this is to say that we're perfect, or that we're not trying to improve things. Just that we're trying to put build system improvements in the proper context amongst all the other work there is to do with our limited resources. It's probably more important that we have support for new Intel client NICs like you would find in a modern desktop system, for example, than it is that we replace make. It's important that we continue to add system calls and libc facilities that other platforms have adopted in order to ease software porting. It's important that we continue to maintain modern JDKs and Python and Go and Rust and C/C++ compilers. It's important that we keep up with security issues and the endless stream of mitigations imposed by the sieve-like nature of speculative CPUs.
There's actually quite a lot of stuff going on for us all the time, and we do still find time to improve the build system. If you have more specifics in mind, that's fantastic and we'd love to here about them concretely! I would encourage you to channel your enthusiasm into writing an illumos project discussion (IPD) describing the issues you see and the work you'd propose to sort them out! You can see some examples of existing IPDs at https://github.com/illumos/ipd
And as ever, if you hit issues in the build as it stands, please file bugs! We can't fix things we haven't heard about.
Cheers.