The article did an ok job of explaining the firecracker limitations they ran into but it was extremely skimpy when it came to qemu and just rushed to the conclusion “we did a lot of work so try our product.”
We replaced Firecracker with QEMU
31–40 of 156 posts
Re: We replaced Firecracker with QEMU
#32I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…
Well that's all nice, but that would also need to be compute-efficient for it to be worthwhile and near-real-time dedupe of memory pages would be a REALLY tough challenge.
Pretty straightforward for disk blocks. Many VM disks are already de-duped, either through snapshopping or through copy on write host filesystems.
The host block cache will end up deduplicating it automatically because all the 'copies' lead back to the same block on disk.
Re: We replaced Firecracker with QEMU
#33I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…
If you already know so much about your application(s), are you sure you need virtualization?
Re: We replaced Firecracker with QEMU
#34The article did an ok job of explaining the firecracker limitations they ran into but it was extremely skimpy when it came to qemu and just rushed to the conclusion “we did a lot of work so try our product.”
yeah I was reading so I could find out what they did.
I understand that they need to sell their product but jeez. don't leave us hanging like that
Re: We replaced Firecracker with QEMU
#35I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…
KVM has KSM (kernel samepage merging) since a long time ago that de-duplicates pages.
But that makes a copy first, and only later notices that the pages are the same and merges them again.
Better to not make copies in the first place.
Re: We replaced Firecracker with QEMU
#36I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…
Doubt it is worth the hassle. How many do you really expect to be identical? An OS isn't large. Your spotify/slack/browser instance is of comparable size. Says more about browser based apps but still.
Basically all code pages should be the same if some other VM has the same version of ubuntu and running the same version of spotify/slack.
And remember that as well as RAM savings, you also get 'instant loading' because there is no need to do slow SSD accesses to load hundreds of megabytes of a chromium binary to get slack running...
Re: We replaced Firecracker with QEMU
#37"the fork was very very bad for eating soup - this is a story about how we migrated to a spoon" ...firecracker does fine what it was designed to - short running fast start workloads. (oh, and the article starts by slightly misusing a bunch of technical terms, firecracker's not technically a hypervisor per se)
..so is it more to support directly deploying functions to the cloud? Like, what AWS Lambda and CloudFront Functions might be built on?
I'm pretty sure firecracker was literally created to underlie AWS Lambda.
EDIT: Okay, https://www.geekwire.com/2018/firecracker-amazon-web-service... says my "pretty sure" memory is in fact correct.
Re: We replaced Firecracker with QEMU
#38I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…
KVM has KSM (kernel samepage merging) since a long time ago that de-duplicates pages.
It has side channel attacks so be careful when enabling: https://pve.proxmox.com/wiki/Kernel_Samepage_Merging_(KSM)
Re: We replaced Firecracker with QEMU
#39"Firecracker's RAM footprint starts low, but once a workload inside allocates RAM, Firecracker will never return it to the host system." Firecracker has a balloon device you can inflate (ie: acquire as much memory inside the VM as possible) and then deflate... returning the memory to the host. You can do this while the VM is running. https://github.com/firecracker-microvm/firecracker/blob/main...
That and the fact that this was after "several weeks of testing" tells me this team doesn't have much virtualization experience. Firecracker is designed to quickly virtualize 1 headless stateless app (like a container), not run hundreds of different programs in a developer environment.
Re: We replaced Firecracker with QEMU
#40"Firecracker's RAM footprint starts low, but once a workload inside allocates RAM, Firecracker will never return it to the host system." Firecracker has a balloon device you can inflate (ie: acquire as much memory inside the VM as possible) and then deflate... returning the memory to the host. You can do this while the VM is running. https://github.com/firecracker-microvm/firecracker/blob/main...
The first footnote says If you squint hard enough, you'll find that Firecracker does support dynamic memory management with a technique called ballooning. However, in practice, it's not usable. To reclaim memory, you need to make sure that the guest OS isn't using it, which, for a general-purpose workload, is nearly impossible