Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced Firecracker with QEMU

hocus.dev

Post reply on HN
Print view View original

141–150 of 156 posts

Re: We replaced Firecracker with QEMU

#141

Post by VWWHFSfQ »

CompuIves wrote: post #89

At CodeSandbox we use Firecracker for hosting development environments, and I agree with the points. Though I don't think that means you should not use Firecracker for running long-lived workloads. We reclaim memory with a memory balloon device, for the disk trimming we discard (& compress) the disk, and for i/o speed we use io_uring (which we only use for scratch disks, the project disks are network disks). It's a t…

I don't know if this is relevant, but I've been intrigued by DragonflyBSD's "vkernel" [0] feature which (supposedly) allows for cloning the entire runtime state of the machine (established TCP connections, etc.) into a completely new userland memory space. I think they use it mostly for kernel debugging right now, but it's interesting to think about the possibilities of being able to just clone an entire running operating system to a new computer without interrupting even a single instruction.

[0] https://www.dragonflybsd.org/docs/handbook/vkernel/

Re: We replaced Firecracker with QEMU

#142

Post by tptacek »

ushakov wrote: post #92
nerpderp82 wrote: post #81

Earlier quoted context omitted.

Shit like this is how you burn customers and then they find out you aren't changing the sheets, they change hotels. Just because you can doesn't mean you should.

It’s a common technique though. I believe it’s called oversubscription, where you rent the same hardware to more tenants hoping they won’t use it all at once. Fly.io themselves admitted they’re oversubscribed and AWS is doing the same for years now Source: https://fly.io/blog/the-serverless-server/

We're not using virtio memory ballooning right now (we've never had the time to get it working).

Re: We replaced Firecracker with QEMU

#143

Post by [deleted] »

mgaunard wrote: post #123

I know a good way to make a process make the most of the hardware and play cooperatively with other processes: don't use virtualization. I will never understand the whole virtual machine and cloud craze. Your operating system is better than any hypervisor at sharing resources efficiently.

[deleted]

Re: We replaced Firecracker with QEMU

#144

Post by Freedom2 »

mgaunard wrote: post #123

I know a good way to make a process make the most of the hardware and play cooperatively with other processes: don't use virtualization. I will never understand the whole virtual machine and cloud craze. Your operating system is better than any hypervisor at sharing resources efficiently.

After reading the README of virtualization tools (and looking at the author) I discovered the benefits of using them. I recommend also giving that a try.

Re: We replaced Firecracker with QEMU

#145

Post by kritr »

londons_explore wrote: post #4

I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…

I believe we do this on Windows for Windows Sandbox. It works well but you will take a hit on performance to do the block resolution compared to always paging into physical memory.

https://learn.microsoft.com/en-us/windows/security/applicati...

Re: We replaced Firecracker with QEMU

#146

Post by mgaunard »

Freedom2 wrote: post #144
mgaunard wrote: post #123

I know a good way to make a process make the most of the hardware and play cooperatively with other processes: don't use virtualization. I will never understand the whole virtual machine and cloud craze. Your operating system is better than any hypervisor at sharing resources efficiently.

After reading the README of virtualization tools (and looking at the author) I discovered the benefits of using them. I recommend also giving that a try.

I do have to use it since someone early on in the company I work at decided to do everything with AWS and Kubernetes.

The fact of the matter is that it's just inefficient, slow and expensive.

Bare metal is simple, fast, and keeps you in control.

Re: We replaced Firecracker with QEMU

#147

Post by datadeft »

mike_hock wrote: post #120
datadeft wrote: post #116

Tl;dr: We tried to misuse technology and we failed. If Firecracker was developed for a single binary executed fir a short period of time why do you try to use it for multiple executables running for a long time? Does it make any sense to even try?

Where in the "sales" pitch on the fancy-CSS website as well as the README does it say only to use it for single-shot workloads? I think the complaints are perfectly valid.

"Firecracker is an alternative to QEMU that is purpose-built for running serverless functions and containers safely and efficiently, and nothing more." [1]

Interesting. I guess we are reading a different website.

1. https://firecracker-microvm.github.io/

Re: We replaced Firecracker with QEMU

#148

Post by freeone3000 »

blinkingled wrote: post #136
freeone3000 wrote: post #11

Earlier quoted context omitted.

It kind of just works. It’s actually broken under Debian 13 for some reason; memory usage infinitely balloons if the feature is enabled.

13 is debian-testing so I guess Microsoft still has time to make it work - last I checked it wasn't yet on Azure supported list.

For context: Most developers run testing since debian-stable averages two years behind

Re: We replaced Firecracker with QEMU

#149

Post by 9659 »

hamandcheese wrote: post #33
londons_explore wrote: post #4

I really want VM's to integrate 'smarter' with the host. For example, if I'm running 5 VM's, there is a good chance that many of the pages are identical. Not only do I want those pages to be deduplicated, but I want them to be zero-copy (ie. not deduplicated after-the-fact by some daemon). To do that, the guest block cache needs to be integrated with the host block-cache, so that whenever some guest application tries…

If you already know so much about your application(s), are you sure you need virtualization?

People run containers for two reasons: #1. They cannot control their devs with python dependencies. #2. Everyone runs containers! Can't be left behind.

Re: We replaced Firecracker with QEMU

#150

Post by CompuIves »

alexeldeib wrote: post #133
CompuIves wrote: post #89

At CodeSandbox we use Firecracker for hosting development environments, and I agree with the points. Though I don't think that means you should not use Firecracker for running long-lived workloads. We reclaim memory with a memory balloon device, for the disk trimming we discard (& compress) the disk, and for i/o speed we use io_uring (which we only use for scratch disks, the project disks are network disks). It's a t…

These blogs are wonderful. I'd read them before figuring out firecracker snapshot/restore, but wanted to say it here. > i/o speed we use io_uring custom io_uring based driver for the VM block devices? or what do you mean here?

Thank you!

> custom io_uring based driver for the VM block devices? or what do you mean here?

We're using the async io backend that's shipped with Firecracker for our scratch disks.

Post reply on HN