Live data from Hacker News

We replaced Firecracker with QEMU

hocus.dev

11–20 of 156 posts

Re: We replaced Firecracker with QEMU

#11

> The main issue we've had with QEMU is that it has too many options you need to configure. For instance, enabling your VM to return unused RAM to the host requires at least three challenging tasks This just works on Hyper-V Linux guests btw. For all the crap MS gets they do some things very right.

It kind of just works. It’s actually broken under Debian 13 for some reason; memory usage infinitely balloons if the feature is enabled.

Re: We replaced Firecracker with QEMU

#12

No mention of Cloud Hypervisor [1]…perhaps they don’t know about it? It’s based in part on Firecracker and supports free page reporting, virtio-blk-pci, PCI passthrough, and (I believe) discard in virtio-blk. [1]: https://www.cloudhypervisor.org/

We do, and we'd love to use it in the future. We've found that it's not ready for prime time yet and it's missing some features. The biggest problem was that it does not support discard operations yet. Here's a short writeup we did about VMMs that we considered: https://github.com/hocus-dev/hocus/blob/main/rfd/0002-worksp...

Re: We replaced Firecracker with QEMU

#13

I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…

KVM has KSM (kernel samepage merging) since a long time ago that de-duplicates pages.

Re: We replaced Firecracker with QEMU

#15

No mention of Cloud Hypervisor [1]…perhaps they don’t know about it? It’s based in part on Firecracker and supports free page reporting, virtio-blk-pci, PCI passthrough, and (I believe) discard in virtio-blk. [1]: https://www.cloudhypervisor.org/

We do, and we'd love to use it in the future. We've found that it's not ready for prime time yet and it's missing some features. The biggest problem was that it does not support discard operations yet. Here's a short writeup we did about VMMs that we considered: https://github.com/hocus-dev/hocus/blob/main/rfd/0002-worksp...

Thanks for the link to the elaboration! FYI footnotes 3 and 4 seem to be swapped.

Re: We replaced Firecracker with QEMU

#16

I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…

Doubt it is worth the hassle. How many do you really expect to be identical?

An OS isn't large. Your spotify/slack/browser instance is of comparable size. Says more about browser based apps but still.

Re: We replaced Firecracker with QEMU

#17

I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…

OpenVZ does this. If you have 5 VMs each loading the same library then memory is conserved, as I understand it.

Re: We replaced Firecracker with QEMU

#18
post #3

Listen people, Firecracker is NOT A HYPERVISOR. A hypervisor runs right on the hardware. KVM is a hypervisor. Firecracker is a process that controls KVM. If you want to call firecracker (and QEMU, when used in conjunction with KVM) a VMM ("virtual machine monitor") I won't complain. But please please please, we need a word for what KVM and Xen are, and "hypervisor" is the best fit. Stop using that word for a user-lev…

Keep fighting the good fight, friend.

Although I’ll note that the line between a VMM and hypervisor are not always clear. E.g., KVM includes some things that other hypervisors delegate to the VMM (such as instruction completion). And macOS’s hypervisor.framework is almost a pass through to the CPU’s raw capabilities.

Re: We replaced Firecracker with QEMU

#19
post #6

Fly uses Firecracker, and they host long-running processes. I wonder what's their opinion about it.

I think their usecase makes a lot of sense as their workloads consume a predefined amount of ram. As a customer you rent a VM with a specified amount of memory so fly.io does not care about reclaiming it from a running VM.

Depends on if they're using smart memory allocation to keep costs lower, IE, if they can pattern that certain workloads only need N amount of memory at Y time, they can effectively borrow memory from one VM for usage in another that has an opposite statistical likelihood of needing that memory.

This is why paying for dedicated memory is often more expensive than its counter part, because that dedicated memory is not considered as part of pooling.

Re: We replaced Firecracker with QEMU

#20
post #3

Listen people, Firecracker is NOT A HYPERVISOR. A hypervisor runs right on the hardware. KVM is a hypervisor. Firecracker is a process that controls KVM. If you want to call firecracker (and QEMU, when used in conjunction with KVM) a VMM ("virtual machine monitor") I won't complain. But please please please, we need a word for what KVM and Xen are, and "hypervisor" is the best fit. Stop using that word for a user-lev…

> virtual machine monitor

Is it good to think of libvirt as a virtual machine mointor, or is that more "virtual machine management"?

Post reply on HN