Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced Firecracker with QEMU

hocus.dev

Post reply on HN
Print view View original

101–110 of 156 posts

Re: We replaced Firecracker with QEMU

#101

Post by crabbone »

Izmaki wrote: post #45
Izmaki wrote: post #41

Earlier quoted context omitted.

I don't know if _one_ such article exists, but here is a piece of tech doc from oVirt (yet another tool) that shows how - or that - VDSM is used by oVirt to communicate with QEMU through libvirt: https://www.ovirt.org/develop/architecture/architecture.html... In really simple terms, so simple that I'm not 100% sure they are correct: * KVM is a hypervisor, or rather it lets you turn linux into a hypervisor [1], which…

Ps: typically if you want to run VMs you are faced with two paths only: 1) you want one or more VMs on your personal computer or 2) you want one or more VMs in an office environment. On the first path you are likely going to be just fine with VirtualBox, VMWare Workstation or Hyper-V (Windows only) / Parallels (Mac intended). Which one you should pick depends on your desired use of the machines. On the second path yo…

> On the first path you are likely going to be just fine with VirtualBox, VMWare Workstation or Hyper-V (Windows only) / Parallels (Mac intended).

As a Linux user, why would you want to use VirtualBox or VMWare Workstation? They are not so well integrated with the system, and, frankly, VirtualBox is more of a toy VM player... just go for virt-manager. It gives a conceptually similar interface to VirtualBox, but better integration with the rest of the system. Especially, when it comes to stuff like sending different key combinations.

I honestly cannot think of a single benefit to using VirtualBox (and I'm less familiar with VMWare player) compared to virt-manager. My guess is that it's more often used because it's also a common choice on MS Windows, so, you get more hits if you are going to search the Web for questions associated to VMs / you'd get tutorials for how to set up a VM that use VirtualBox. But, if you apply yourself to learning how either one of these works, you'd see no reason to choose it.

Re: We replaced Firecracker with QEMU

#102

Post by cthalupa »

compscigirl wrote: post #95
veber-alex wrote: post #76

Earlier quoted context omitted.

KVM is a type-1 hypervisor [1] [1]: https://www.redhat.com/en/topics/virtualization/what-is-KVM

Not really, calling KVM a type-1 is a misunderstanding of what the “bare-metal” distinction is referring to. The real difference between the two types is whether the hypervisor owns the hardware or not. In the case of a type-1, the hypervisor runs below the kernel and controls access to the hardware, even for the kernel. In type-2, the hypervisor runs on the kernel, which owns the hardware, and must go through the ke…

But that's not how that works. KVM is as "bare-metal" in access to the system as ESXi is. The hypervisor code runs in ring 0 in both cases.

Re: We replaced Firecracker with QEMU

#103

Post by imtringued »

geodel wrote: post #98
nerpderp82 wrote: post #81

Earlier quoted context omitted.

Shit like this is how you burn customers and then they find out you aren't changing the sheets, they change hotels. Just because you can doesn't mean you should.

>.. they change hotels And then they learn all hotels are doing exactly same thing. One hotel doing is risk, all hotels doing is industry standard. Airlines, hotels, restaurants, doctors and so on oversubscribe all the time. Whoever complains are free to move and add to their further disappointments.

Memory (DDR4) is like 1.50€ per GB nowadays. There is no need to skimp on it. The most expensive component is still the CPU if you actually want to do work and not just idle your webserver 99% of the time.

Re: We replaced Firecracker with QEMU

#104

Post by no_wizard »

nerpderp82 wrote: post #81
no_wizard wrote: post #19

Earlier quoted context omitted.

Depends on if they're using smart memory allocation to keep costs lower, IE, if they can pattern that certain workloads only need N amount of memory at Y time, they can effectively borrow memory from one VM for usage in another that has an opposite statistical likelihood of needing that memory. This is why paying for dedicated memory is often more expensive than its counter part, because that dedicated memory is not…

Shit like this is how you burn customers and then they find out you aren't changing the sheets, they change hotels. Just because you can doesn't mean you should.

Super common though. Everyone from Hetzner to AWS does this, unless you specifically pay for full allocation, which is typically ~2x the price.

Chances are you are using systems that do this and you haven't even noticed.

Re: We replaced Firecracker with QEMU

#105

Post by no_wizard »

imtringued wrote: post #103
geodel wrote: post #98

Earlier quoted context omitted.

>.. they change hotels And then they learn all hotels are doing exactly same thing. One hotel doing is risk, all hotels doing is industry standard. Airlines, hotels, restaurants, doctors and so on oversubscribe all the time. Whoever complains are free to move and add to their further disappointments.

Memory (DDR4) is like 1.50€ per GB nowadays. There is no need to skimp on it. The most expensive component is still the CPU if you actually want to do work and not just idle your webserver 99% of the time.

this ignores just how competitive these services have become and how much competition there is in this marketplace. Anything that allows a company to balance user experience with cost of hardware will be evaluated. The fact is this is super common because even though RAM is super cheap, at scale, these things add up. If you as a provider don't have to spend 120K on RAM and can use that on more CPUs or something else, it can mean having a competitive edge over the provider that does have to spend that 120K.

Re: We replaced Firecracker with QEMU

#106

Post by klabb3 »

nerpderp82 wrote: post #81
no_wizard wrote: post #19

Earlier quoted context omitted.

Depends on if they're using smart memory allocation to keep costs lower, IE, if they can pattern that certain workloads only need N amount of memory at Y time, they can effectively borrow memory from one VM for usage in another that has an opposite statistical likelihood of needing that memory. This is why paying for dedicated memory is often more expensive than its counter part, because that dedicated memory is not…

Shit like this is how you burn customers and then they find out you aren't changing the sheets, they change hotels. Just because you can doesn't mean you should.

Almost everything is oversubscribed in our world.

Malloc will happily “return” the 15 TiB you asked for.

If 10000 people called 911 at the same time, only a tiny fraction would get through (and even fewer would get help).

Evacuating a large city by road would result in giant traffic jams.

There are 5-8x as many parking spots as there are cars (and we still can’t find a goddamn spot).

And of course… the great toilet paper shortage of 2020.

Re: We replaced Firecracker with QEMU

#107

Post by zbrozek »

bonzini wrote: post #52
zbrozek wrote: post #26

Earlier quoted context omitted.

I'd love to get a clear explanation of what libvirt actually does. As far as I can tell it's a qemu argument assembler and launcher. For my own use-case, I just launch qemu from systemd unit files: https://wiki.archlinux.org/title/QEMU#With_systemd_service

The main important point is that Libvirt takes care of privilege separation. It makes sure that if your VM and/or QEMU are broken out of, there are extra layers to prevent getting access to the whole physical machine. For example it runs QEMU as a very limited user and, if you're using SELinux, the QEMU process can hardly read any file other than the vm image file. By contrast the method in the arch wiki runs QEMU as…

Thank you, this is the best explanation of the utility that I've seen. I appreciate it.

Re: We replaced Firecracker with QEMU

#108

Post by mulmen »

nerpderp82 wrote: post #81
no_wizard wrote: post #19

Earlier quoted context omitted.

Depends on if they're using smart memory allocation to keep costs lower, IE, if they can pattern that certain workloads only need N amount of memory at Y time, they can effectively borrow memory from one VM for usage in another that has an opposite statistical likelihood of needing that memory. This is why paying for dedicated memory is often more expensive than its counter part, because that dedicated memory is not…

Shit like this is how you burn customers and then they find out you aren't changing the sheets, they change hotels. Just because you can doesn't mean you should.

I don't see the problem. Is this not the value proposition of the cloud? At scale things like block level de-duplication and over-provisioning of compute are more efficient to say nothing of power and cooling. This efficiency drives down cost. As long as your data isn't leaking and you get the hardware you pay for when you need it what's the problem?

Re: We replaced Firecracker with QEMU

#109

Post by tedunangst »

paulv wrote: post #40
rgbrenner wrote: post #28

"Firecracker's RAM footprint starts low, but once a workload inside allocates RAM, Firecracker will never return it to the host system." Firecracker has a balloon device you can inflate (ie: acquire as much memory inside the VM as possible) and then deflate... returning the memory to the host. You can do this while the VM is running. https://github.com/firecracker-microvm/firecracker/blob/main...

The first footnote says If you squint hard enough, you'll find that Firecracker does support dynamic memory management with a technique called ballooning. However, in practice, it's not usable. To reclaim memory, you need to make sure that the guest OS isn't using it, which, for a general-purpose workload, is nearly impossible

I'm struggling to understand how qemu with free page reporting isn't exactly the same as a firecracker balloon.
Post reply on HN