Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced Firecracker with QEMU

hocus.dev

Post reply on HN
Print view View original

121–130 of 156 posts

Re: We replaced Firecracker with QEMU

#121

Post by dathinab »

paulv wrote: post #40
rgbrenner wrote: post #28

"Firecracker's RAM footprint starts low, but once a workload inside allocates RAM, Firecracker will never return it to the host system." Firecracker has a balloon device you can inflate (ie: acquire as much memory inside the VM as possible) and then deflate... returning the memory to the host. You can do this while the VM is running. https://github.com/firecracker-microvm/firecracker/blob/main...

The first footnote says If you squint hard enough, you'll find that Firecracker does support dynamic memory management with a technique called ballooning. However, in practice, it's not usable. To reclaim memory, you need to make sure that the guest OS isn't using it, which, for a general-purpose workload, is nearly impossible

> is nearly impossible

for many mostly "general purpose" use cases it's quite viable, or else ~fly.io~ AWS Fargate wouldn't be able to use it

this doesn't mean it's easy to implement the necessary automatized tooling etc.

so it's depending on your dev resources and priorities it might be a bad choice

still I feel the article was had quite a bit a being subtil judgemental while moving some quite relevant parts for the content of the article into a footnote and also omitting that this "supposedly unusable tool" is used successfully by various other companies...

like as it it was written by and engineer being overly defensive about their decision due having to defend it to the 100th time because shareholders, customers, higher level management just wouldn't shut up about "but that uses Firecracker"

Re: We replaced Firecracker with QEMU

#122

Post by dathinab »

lessaligned wrote: post #24

"the fork was very very bad for eating soup - this is a story about how we migrated to a spoon" ...firecracker does fine what it was designed to - short running fast start workloads. (oh, and the article starts by slightly misusing a bunch of technical terms, firecracker's not technically a hypervisor per se)

it's not that simple many other companies running longer running jobs, including their competition, use Firecracker

so while Firecracker was designed for thing running just a few seconds there are many places running it with jobs running way longer then that

the problem is if you want to make it work with long running general purpose images you don't control you have to put a ton of work into making it work nicely on all levels of you infrastructure and code ... which is costly ... which a startup competing on a online dev environment compared to e.g. a vm hosting service probably shouldn't wast time on

So AFIK the decision in the article make sense the reasons but listed for the decision are oversimplified to a point you could say they aren't quite right. Idk. why, could be anything from the engineer believing that to them avoiding issues with some shareholder/project lead which is obsessed with "we need to do Firecracker because competition does so too".

Re: We replaced Firecracker with QEMU

#123

Post by mgaunard »

I know a good way to make a process make the most of the hardware and play cooperatively with other processes: don't use virtualization.

I will never understand the whole virtual machine and cloud craze. Your operating system is better than any hypervisor at sharing resources efficiently.

Re: We replaced Firecracker with QEMU

#124

Post by dathinab »

sheepscreek wrote: post #30
lessaligned wrote: post #24

"the fork was very very bad for eating soup - this is a story about how we migrated to a spoon" ...firecracker does fine what it was designed to - short running fast start workloads. (oh, and the article starts by slightly misusing a bunch of technical terms, firecracker's not technically a hypervisor per se)

..so is it more to support directly deploying functions to the cloud? Like, what AWS Lambda and CloudFront Functions might be built on?

yes, it was created originally for AWS Lambda

mainly it's optimized to run code only shortly (init time max 10s, max usage is 15min, and default max request time 130s AFIK)

also it's focused on thin server less functions, like e.g. deserialize some request, run some thin simple business logic and then delegate to other lambdas based on it. This kind of functions often have similar memory usage per-call and if a call is an outlier it can just discard the VM instance soon after (i.e. at most after starting up a new instance, i.e. at most 10s later)

Re: We replaced Firecracker with QEMU

#125

Post by akira2501 »

mgaunard wrote: post #123

I know a good way to make a process make the most of the hardware and play cooperatively with other processes: don't use virtualization. I will never understand the whole virtual machine and cloud craze. Your operating system is better than any hypervisor at sharing resources efficiently.

> cloud craze.

Automatic scaling is great. Cloud parallelization (a.k.a fork) is absolutely wild once you get it rolling. Code deployments are incredibly simple. Never having to worry about physical machines or variable traffic loads is worth the small overhead they charge me for the wrapper. The generic system wide permissions model is an absolute joy once you get over the learning curve.

Re: We replaced Firecracker with QEMU

#126

Post by rob-olmos »

yokaze wrote: post #113
no_wizard wrote: post #104

Earlier quoted context omitted.

Super common though. Everyone from Hetzner to AWS does this, unless you specifically pay for full allocation, which is typically ~2x the price. Chances are you are using systems that do this and you haven't even noticed.

Werner Vogels seems to disagree: https://twitter.com/Werner/status/25137574680

Tweeted Sep 21, 2010, might still be true for EC2 VMs though.

Re: We replaced Firecracker with QEMU

#127

Post by zokier »

datadeft wrote: post #116

Tl;dr: We tried to misuse technology and we failed. If Firecracker was developed for a single binary executed fir a short period of time why do you try to use it for multiple executables running for a long time? Does it make any sense to even try?

AWS uses firecracker to execute long-running Fargate tasks; its hardly misuse

Re: We replaced Firecracker with QEMU

#128

Post by favflam »

I know that Firecracker does not let you bind mount volumes, but QEMU does. So, we changed to QEMU from Firecracker. If you run the workloads in Kubernetes, you just have to change a single value in a yaml file to change the runtime.

I would be scared to let unknown persons use QEMU that bind mounts volumes as that is a huge security risk. Firecracker, I think, was designed from the start to run un-sanitized workloads, hence, no bind mounting.

Re: We replaced Firecracker with QEMU

#129

Post by brunoborges »

favflam wrote: post #128

I know that Firecracker does not let you bind mount volumes, but QEMU does. So, we changed to QEMU from Firecracker. If you run the workloads in Kubernetes, you just have to change a single value in a yaml file to change the runtime. I would be scared to let unknown persons use QEMU that bind mounts volumes as that is a huge security risk. Firecracker, I think, was designed from the start to run un-sanitized workload…

> you just have to change a single value in a yaml file

Most dangerous 12-words sentence.

Re: We replaced Firecracker with QEMU

#130

Post by kdmytro »

mgaunard wrote: post #123

I know a good way to make a process make the most of the hardware and play cooperatively with other processes: don't use virtualization. I will never understand the whole virtual machine and cloud craze. Your operating system is better than any hypervisor at sharing resources efficiently.

I want to segregate, not to share.
Post reply on HN