> QEMU is often the subject of bugs affecting its reliability and security. {{citation needed}}? When I ran the numbers in 2019, there hadn't been guest exploitable vulnerabilities that affected devices normally used for IaaS for 3 years. Pretty much every cloud outside the big three (AWS, GCE, Azure) runs on QEMU. Here's a talk I gave about it that includes that analysis: slides - https://kvm-forum.qemu.org/2019/kvm…
Why Oxide Chose Illumos
71–80 of 177 posts
Re: Why Oxide Chose Illumos
#72Given all of that, and taking into account building a product on top of it, and thus needing to support it and stand behind it, Linux wasn't the best choice. Looking ahead (in terms of decades) and not just shipping a product now, it was found that an alternate ecosystem existed to support that.
Culture of the community, design principles, maintainability are all things to consider beyond just "is it popular".
Exciting times in computing once again!
Re: Why Oxide Chose Illumos
#73> Xen: Large and complicated (by dom0) codebase, discarded for KVM by AMZN 1. Xen Type-1 hypervisor is smaller than KVM/QEMU. 2. Xen "dom0" = Linux/FreeBSD/OpenSolaris. KVM/bhyve also need host OS. 3. AMZN KVM-subset: x86 cpu/mem virt, blk/net via Arm Nitro hardware. 4. bhyve is Type-2. 5. Xen has Type-2 (uXen). 6. Xen dom0/host can be disaggregated (Hyperlaunch), unlike KVM. 7. pKVM (Arm/Android) is smaller than KVM…
Talking about "technological and political issues" without mentioning any, or without mentioning which components would need to be revived, sounds a lot like FUD unfortunately. Mixing and matching traditional and systemd components is super common, for example Fedora and RHEL use chrony instead of timesyncd, and NetworkManager instead of networkd.
I don't know why you think none were mentioned - to name one, they link a GitHub issue created against the systemd repository by a Googler complaining that systemd is inappropriately using Google's NTP servers, which at the time were not a public service, and kindly asking for systemd to stop using them.
This request was refused and the issue was closed and locked.
Behaviour like this from the systemd maintainers can only appear bizarre, childish, and unreasonable to any unprejudiced observer, putting their character and integrity into question and casting doubt on whether they should be trusted with the maintenance of software so integral to at least a reasonably large minority of modern Linux systems.
Re: Why Oxide Chose Illumos
#74Earlier quoted context omitted.
> When I ran the numbers in 2019, there hadn't been guest exploitable vulnerabilities that affected devices normally used for IaaS for 3 years. So there existed known guest-exploitable vulnerabilities as recently as 8 years ago. Maybe that, combined with the fact that QEMU is not written in Rust, is what is causing Oxide to decide against QEMU. I think it's fair to say that any sufficiently large codebase originally…
That would be a damn good record though, isn't it? (I am fairly sure that more were found since, but the point is that these are pretty rare). Firecracker, which is written in Rust, had one in 2019: https://www.cve.org/CVERecord?id=CVE-2019-18960 Also QEMU's fuzzing is very sophisticated. Most recent vulnerabilities were found that way rather than by security researchers, which I don't think it's the case for "compet…
Re: Why Oxide Chose Illumos
#75These sound like reason you retconned so it sounds like you didn't choose Illumos because your founder used to work at Sun and Joyent before. :-) Frankly I don't understand why they blogged that at all. It reeks of desperation, like they feel they need to defend their choice. They don't. It also should not matter to their customers. They get exposed APIs and don't have to care about the implementation details.
It's not a blog post, it's an RFD. We have a strong focus on writing as part of thinking and making decisions, and when we can, we like to publish our decision making documents in the spirit of open source. This is not a defence of our position so much as a record of the process through which we arrived at it. This is true of our other RFDs as well, which you can see on the site there. > It also should not matter to…
Re: Why Oxide Chose Illumos
#76Earlier quoted context omitted.
Early Oxide founders came from Joyent which was an illumos shop and Cantrill is quite vocal about the history of Solaris, OpenSolaris, and illumos.
> Joyent which was an illumos shop And before that, they used to run FreeBSD. Mentioned for example in this comment by Bryan Cantrill a decade ago: https://news.ycombinator.com/item?id=6254092 > […] Speaking only for us (I work for Joyent), we have deployed hundreds of thousands of zones into production over the years -- and Joyent was running with FreeBSD jails before that […] And I’ve seen some other primary source…
As I recall they were also the original host of Twitter, which if I recall was Rails back in the day.
Re: Why Oxide Chose Illumos
#77> To mitigate all this, we’re intending to stick with the OSS build, which includes no CCL code.
[0] https://news.ycombinator.com/item?id=41256222
[1] https://rfd.shared.oxide.computer/rfd/0110Re: Why Oxide Chose Illumos
#78> QEMU is often the subject of bugs affecting its reliability and security. {{citation needed}}? When I ran the numbers in 2019, there hadn't been guest exploitable vulnerabilities that affected devices normally used for IaaS for 3 years. Pretty much every cloud outside the big three (AWS, GCE, Azure) runs on QEMU. Here's a talk I gave about it that includes that analysis: slides - https://kvm-forum.qemu.org/2019/kvm…
What do the big three use?
Google – "KVM-based hypervisor"
Azure – Hyper-V
You can of course assume that all of them heavily customize the underlying implemenation for their own needs and for their own hardware. And then they have stuff like Firecracker, GVisor etc. layered on top depending on the product line.
Re: Why Oxide Chose Illumos
#79Been running Bhyve on FreeBSD (technically FreeNAS). Found PCIe pass-through of NMVe drives was fairly straight forward once the correct incantations were found, but network speed to host has been fairly abysmal. On my admittedly aging Threadripper 1920X, I can only get ~2-3 Gbps peak from a Linux guest. That's with virtio, the virtual intel "card" is even slower. They went with Illumos though, so curious if the poor…
It's been a minute since I messed with bhyve on FreeBSD, but I'm pretty sure you have to switch out the networking stack to something like Netgraph if you intend to use fast networking.
Re: Why Oxide Chose Illumos
#80> QEMU is often the subject of bugs affecting its reliability and security. {{citation needed}}? When I ran the numbers in 2019, there hadn't been guest exploitable vulnerabilities that affected devices normally used for IaaS for 3 years. Pretty much every cloud outside the big three (AWS, GCE, Azure) runs on QEMU. Here's a talk I gave about it that includes that analysis: slides - https://kvm-forum.qemu.org/2019/kvm…
QEMU typically uses KVM for the hypervisor, so the vulnerabilities will be KVM anyway. The big three all use KVM now. Oxide decided to go with bhyve instead of KVM.