Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced Firecracker with QEMU

hocus.dev

Post reply on HN
Print view View original

41–50 of 156 posts

Re: We replaced Firecracker with QEMU

#41

Post by Izmaki »

arun-mani-j wrote: post #23
gwd wrote: post #3

Listen people, Firecracker is NOT A HYPERVISOR. A hypervisor runs right on the hardware. KVM is a hypervisor. Firecracker is a process that controls KVM. If you want to call firecracker (and QEMU, when used in conjunction with KVM) a VMM ("virtual machine monitor") I won't complain. But please please please, we need a word for what KVM and Xen are, and "hypervisor" is the best fit. Stop using that word for a user-lev…

I think you could help me answer the question that has been in my mind for a month :) Is there any article that tells the difference and relationship between KVM, QEMU, libvirt, virt-manager, Xen, Proxmox etc. with their typical use cases?

I don't know if _one_ such article exists, but here is a piece of tech doc from oVirt (yet another tool) that shows how - or that - VDSM is used by oVirt to communicate with QEMU through libvirt: https://www.ovirt.org/develop/architecture/architecture.html...

In really simple terms, so simple that I'm not 100% sure they are correct:

* KVM is a hypervisor, or rather it lets you turn linux into a hypervisor [1], which will let you run VMs on your machine. I've heard KVM is rather hard to work with (steep learning curve). (Xen is also a hypervisor.)

* QEMU is a wrapper-of-a-sorts (a "machine emulator and virtualizer" [2]) which can be used on top of KVM (or Xen). "When used as a virtualizer, QEMU achieves near native performance by executing the guest code directly on the host CPU. QEMU supports virtualization when executing under the Xen hypervisor or using the KVM kernel module in Linux." [2]

* libvirt "is a toolkit to manage virtualization platforms" [3] and is used, e.g., by VDSM to communicate with QEMU.

* virt-manager is "a desktop user interface for managing virtual machines through libvirt" [4]. The screenshots on the project page should give an idea of what its typical use-case is - think VirtualBox and similar solutions.

* Proxmox is the above toolstack (-ish) but as one product.

---

[1] https://www.redhat.com/en/topics/virtualization/what-is-KVM

[2] https://wiki.qemu.org/Main_Page

[3] https://libvirt.org/

[4] https://virt-manager.org/

Re: We replaced Firecracker with QEMU

#42

Post by MuffinFlavored »

zbrozek wrote: post #26
MuffinFlavored wrote: post #20

Earlier quoted context omitted.

> virtual machine monitor Is it good to think of libvirt as a virtual machine mointor, or is that more "virtual machine management"?

I'd love to get a clear explanation of what libvirt actually does. As far as I can tell it's a qemu argument assembler and launcher. For my own use-case, I just launch qemu from systemd unit files: https://wiki.archlinux.org/title/QEMU#With_systemd_service

> As far as I can tell it's a qemu argument assembler

Raises a few questions to me:

Can you use KVM/do KVM stuff without QEMU?

Can you do libvirt stuff without QEMU?

Hoping the answers to both aren't useless/"technically, but why would you want to?"

Re: We replaced Firecracker with QEMU

#43

Post by jacobr1 »

arun-mani-j wrote: post #23
gwd wrote: post #3

Listen people, Firecracker is NOT A HYPERVISOR. A hypervisor runs right on the hardware. KVM is a hypervisor. Firecracker is a process that controls KVM. If you want to call firecracker (and QEMU, when used in conjunction with KVM) a VMM ("virtual machine monitor") I won't complain. But please please please, we need a word for what KVM and Xen are, and "hypervisor" is the best fit. Stop using that word for a user-lev…

I think you could help me answer the question that has been in my mind for a month :) Is there any article that tells the difference and relationship between KVM, QEMU, libvirt, virt-manager, Xen, Proxmox etc. with their typical use cases?

[flagged]

Re: We replaced Firecracker with QEMU

#44

Post by theossuary »

arun-mani-j wrote: post #23
gwd wrote: post #3

Listen people, Firecracker is NOT A HYPERVISOR. A hypervisor runs right on the hardware. KVM is a hypervisor. Firecracker is a process that controls KVM. If you want to call firecracker (and QEMU, when used in conjunction with KVM) a VMM ("virtual machine monitor") I won't complain. But please please please, we need a word for what KVM and Xen are, and "hypervisor" is the best fit. Stop using that word for a user-lev…

I think you could help me answer the question that has been in my mind for a month :) Is there any article that tells the difference and relationship between KVM, QEMU, libvirt, virt-manager, Xen, Proxmox etc. with their typical use cases?

KVM is a Linux kernel implementation of the cpu extensions to accelerate vms to near bare metal speeds.

Qemu is a user space system emulator. It can emulate in software different architectures like ARM, x86, etc. It can also emulate drivers, networking, disks, etc. Is called via the command line.

The reason you'll see Qemu/KVM a lot is because Qemu is the emulator, the things actually running the VM. And it utilizes KVM (on linux, OSX has HVF, for example) to accelerate the VM when the host architecture matches the VM's.

Libvirt is an XML based API on top of Qemu (and others). It allows you to define networks, VMs (it calls them domains), and much more with a unified XML schema through libvirtd.

Virsh is a CLI tool to manage libvirtd. Virt-manager is a GUI to do the same.

Proxmox is Debian under the hood with Qemu/KVM running VMs. It provides a robust web UI and easy clustering capabilities. Along with nice to haves like easy management of disks, ceph, etc. You can also manage Ceph through an API with Terraform.

Xen is an alternative hypervisor (like esxi). Instead of running on top of Linux, Xen has it's own microkernel. This means less flexibility (there's no Linux body running things), but also simpler to manage and less attack surface. I haven't played much with xen though, KVM is kind of the defacto, but IIRC AWS used to use a modified Xen before KVM came along and ate Xen's lunch.

Re: We replaced Firecracker with QEMU

#45

Post by Izmaki »

Izmaki wrote: post #41
arun-mani-j wrote: post #23

Earlier quoted context omitted.

I think you could help me answer the question that has been in my mind for a month :) Is there any article that tells the difference and relationship between KVM, QEMU, libvirt, virt-manager, Xen, Proxmox etc. with their typical use cases?

I don't know if _one_ such article exists, but here is a piece of tech doc from oVirt (yet another tool) that shows how - or that - VDSM is used by oVirt to communicate with QEMU through libvirt: https://www.ovirt.org/develop/architecture/architecture.html... In really simple terms, so simple that I'm not 100% sure they are correct: * KVM is a hypervisor, or rather it lets you turn linux into a hypervisor [1], which…

Ps: typically if you want to run VMs you are faced with two paths only: 1) you want one or more VMs on your personal computer or 2) you want one or more VMs in an office environment.

On the first path you are likely going to be just fine with VirtualBox, VMWare Workstation or Hyper-V (Windows only) / Parallels (Mac intended). Which one you should pick depends on your desired use of the machines.

On the second path you would go with a solution that deals with the nitty-gritty details, such as Proxmox, oVirt, Hyper-V, ESXi, or any of the other many available options - granted you are not going full cloud-based, which opens up a whole lot of different options too.

You would generally never need to worry about which components are needed where and why. I've had to worry about it once or twice before, because I've had to debug why an oVirt solution was not behaving like I wanted it to behave. Knowing the inner workings helps in that case.

Re: We replaced Firecracker with QEMU

#46

Post by theossuary »

jacobr1 wrote: post #43
arun-mani-j wrote: post #23

Earlier quoted context omitted.

I think you could help me answer the question that has been in my mind for a month :) Is there any article that tells the difference and relationship between KVM, QEMU, libvirt, virt-manager, Xen, Proxmox etc. with their typical use cases?

[flagged]

Damn, outdone by ChatGPT

Re: We replaced Firecracker with QEMU

#47

Post by heavyset_go »

Someone posted this and then immediately deleted their comment: https://qemu.readthedocs.io/en/latest/system/i386/microvm.ht...

I didn't know it existed until they posted, but QEMU has a Firecracker-inspired target:

> microvm is a machine type inspired by Firecracker and constructed after its machine model.

> It’s a minimalist machine type without PCI nor ACPI support, designed for short-lived guests. microvm also establishes a baseline for benchmarking and optimizing both QEMU and guest operating systems, since it is optimized for both boot time and footprint.

Re: We replaced Firecracker with QEMU

#48

Post by [deleted] »

paulv wrote: post #40
rgbrenner wrote: post #28

"Firecracker's RAM footprint starts low, but once a workload inside allocates RAM, Firecracker will never return it to the host system." Firecracker has a balloon device you can inflate (ie: acquire as much memory inside the VM as possible) and then deflate... returning the memory to the host. You can do this while the VM is running. https://github.com/firecracker-microvm/firecracker/blob/main...

The first footnote says If you squint hard enough, you'll find that Firecracker does support dynamic memory management with a technique called ballooning. However, in practice, it's not usable. To reclaim memory, you need to make sure that the guest OS isn't using it, which, for a general-purpose workload, is nearly impossible

[deleted]

Re: We replaced Firecracker with QEMU

#49

Post by bonzini »

reactordev wrote: post #27
arun-mani-j wrote: post #23

Earlier quoted context omitted.

I think you could help me answer the question that has been in my mind for a month :) Is there any article that tells the difference and relationship between KVM, QEMU, libvirt, virt-manager, Xen, Proxmox etc. with their typical use cases?

KVM is kernel-based virtual machine, with libvirt being its API abstraction over all of it. QEMU is a virtual machine host that leverages kvm or software virtualization to spin up machines on the host. virt-manager does the same. Xen is another virtual machine host, like KVM. Proxmox is a virtual machine manager (like QEMU, virt-manager) but is web based. Libvirt will provide abstraction for kvm,qemu,xen Use cases: p…

This is almost entirely wrong especially as far as QEMU, Libvirt and virt-manager are concerned.

QEMU is a low level process that represents the virtual machine. It has no equivalent in Xen. Using QEMU directly is not a good idea unless your needs for VM configurations change all the time and you hardly reuse VMs.

Libvirt is at a higher level than QEMU. It manages the QEMU processes and gives them access to system resources (image files, network interfaces, pass-through PCI devices). It also makes it easy to manage the configuration of your virtual machines and the resources they use.

Higher still is virt-manager, which is a GUI interface for libvirt. Proxmox sits at roughly the same level as virt-manager.

Re: We replaced Firecracker with QEMU

#50

Post by simcop2387 »

MuffinFlavored wrote: post #42
zbrozek wrote: post #26

Earlier quoted context omitted.

I'd love to get a clear explanation of what libvirt actually does. As far as I can tell it's a qemu argument assembler and launcher. For my own use-case, I just launch qemu from systemd unit files: https://wiki.archlinux.org/title/QEMU#With_systemd_service

> As far as I can tell it's a qemu argument assembler Raises a few questions to me: Can you use KVM/do KVM stuff without QEMU? Can you do libvirt stuff without QEMU? Hoping the answers to both aren't useless/"technically, but why would you want to?"

> Can you use KVM/do KVM stuff without QEMU?

Yes there's a few things out there like Firecracker that use KVM without using QEMU. I'm not completely aware of all of them but they do exist

> Can you do libvirt stuff without QEMU?

Yes it can also manager LXC containers and a few other types like Xen and Bhyve and Virtuozzo, like QEMU without KVM. The without KVM part is important to letting you run VMs that are emulating other architectures than the native one.

For a good bit of this, it is "why would you want to" but there are definitely real cases where you'd want to be able to do this. Like the LXC or Virtuozzo support means that you can run lighter weight containers (same underlying tech as Docker essentially) through the same orchestration/management that you use for virtual machines. And the Bhyve support lets you do the same thing for running things on top of FreeBSD (though I've never used it this way) so that a heterogeneous mix of hosts is managed through the same interfaces.

Post reply on HN