Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ignite – Use Firecracker VMs with Docker images

github.com

Post reply on HN
Print view View original

71–80 of 84 posts

Re: Ignite – Use Firecracker VMs with Docker images

#71

Post by 0xbadcafebee »

tptacek wrote: post #65
0xbadcafebee wrote: post #33

Earlier quoted context omitted.

> All serverside cloud-style VMs get tap devices Unless you want to PCI pass-through an SRIOV VF into a guest? If you're paying out the ass for Cisco cloud gear you probably want the guest to have direct access to a card... Again, I don't remember the specifics, but depending on the underlying networking (on the host etc), there may be some issues with IPSec (but I could be wrong). From some random searching: https:/…

Yep, PCI pass-through is 100% something you can't do with Firecracker. To me, that's a perfect example of a feature that's not really there to support cloud-style serverside workloads, but that might just be my bias.

I think the other way to look at it is there's just different workloads that need different solutions. Firecracker is perfect for Lamba-style workloads.

But some companies (who don't want to manage datacenter resources, but do want the large scale and flexibility of cloud computing) want a whiz-bang multi-tenant enterprise containerized solution (say, using K8s as orchestrator). And maybe one of their critical applications has certain requirements that necessitate an HBA, deep packet inspection, an HSM, a high-precision clock, etc. They need strong isolation guarantees, so they want a VM, but they want K8s to manage it.

Kata Containers seems to fit the latter, as it can change hypervisors, supports more architectures, advanced networking features, device assignment, etc. Seems to align most with on-prem cloud as it's maintained by the OpenStack people.

Re: Ignite – Use Firecracker VMs with Docker images

#72

Post by NathanKP »

nailer wrote: post #54

Can someone help me understand whether this is: - My app per the Dockerfile - Container - Firecracker VM Or: - My app per the Dockerfile - Firecracker VM Thanks!

The latter. If you typed `docker ps` on your host there would be nothing showing up. The application is running inside of a micro VM.

It should also be noted that containerized processes that are running on a host will still show up on the host's process tree if you look at the output of `ps`, `htop` or similar commands. Containers aren't real boundaries, just namespaces on the host machine. This is the not the case when using Ignite, as the process from the container image is running in an actual micro VM, rather than running on the host inside of a namespace

Re: Ignite – Use Firecracker VMs with Docker images

#73

Post by mbreese »

Atreiden wrote: post #51
mbreese wrote: post #7

For me, this is the deal breaker, but this is still exciting to me: > Note: At the moment ignite and ignited need root privileges on the host to operate due to certain operations (e.g. mount). This will change in the future. I’d love it if this ends up getting changed. It’s a hard line to walk, but this would make Docker containers feasible on multi-user systems. (Without using Singularity, which has its own downside…

Found the HPC guy. I was so excited for this until I got to that line. I'll still be following this closely though, the potential is huge.

So, you’ve been waiting for something like this too?

I’d love to be able to submit Slurm jobs backed by a Docker container. That would be a reproducibility dream for me. And make it so I didn’t have to install so much software or still maintain Environment modules.

As it is, I can get partially there with Singularity, which isn’t terrible.

Re: Ignite – Use Firecracker VMs with Docker images

#74

Post by mbreese »

regularfry wrote: post #60
nik736 wrote: post #52

Earlier quoted context omitted.

Just spawn a new VM on a hypervisor and delete the old VM on the hypervisor that is dying. You can also balance load this way.

Assume any suggestion starting with "just" is equally obvious to the operators as it is to you. In this case, it would be helpful to start from the observation that live migration is non-negotiable, which immediately rules out any form of "turn it on then off again."

In this case, I think the “just” is justified. (Normally I’d agree with you)

If you’re starting from the premise that live migration is non-negotiable, then I’d argue that you likely don’t need a quick start. Long lived services can afford a slower start using a different VM technology. We’re not talking hours to provision a server, but the difference between 100ms and a few minutes.

My question to you is — what kind of service would need to start fast and be able to migrate? I just don’t see that Venn diagram having a big overlap. But maybe I’m wrong…

Re: Ignite – Use Firecracker VMs with Docker images

#75

Post by Atreiden »

mbreese wrote: post #73
Atreiden wrote: post #51

Earlier quoted context omitted.

Found the HPC guy. I was so excited for this until I got to that line. I'll still be following this closely though, the potential is huge.

So, you’ve been waiting for something like this too? I’d love to be able to submit Slurm jobs backed by a Docker container. That would be a reproducibility dream for me. And make it so I didn’t have to install so much software or still maintain Environment modules. As it is, I can get partially there with Singularity, which isn’t terrible.

Absolutely. With firecracker, AWS seems to have solved the multi-tenancy problem that has plagued HPC since its inception.

I've just been waiting for some way to implement that in a sane way for on-prem infrastructure. If not for the root requirement, this would be perfect.

Singularity does get us part of the way there, but it's just another abstraction layer that researchers have to work around. And it's far from perfect.

Re: Ignite – Use Firecracker VMs with Docker images

#76

Post by tptacek »

0xbadcafebee wrote: post #71
tptacek wrote: post #65

Earlier quoted context omitted.

Yep, PCI pass-through is 100% something you can't do with Firecracker. To me, that's a perfect example of a feature that's not really there to support cloud-style serverside workloads, but that might just be my bias.

I think the other way to look at it is there's just different workloads that need different solutions. Firecracker is perfect for Lamba-style workloads . But some companies (who don't want to manage datacenter resources, but do want the large scale and flexibility of cloud computing) want a whiz-bang multi-tenant enterprise containerized solution (say, using K8s as orchestrator). And maybe one of their critical appli…

What you're calling a "Lambda-style workload" really seems to capture >95% of the EC2-style workload.

Re: Ignite – Use Firecracker VMs with Docker images

#77

Post by expertsteve »

tillk wrote: post #41
duskwuff wrote: post #25

I like the way this sounds, but... is the project still alive? There haven't been any commits to the main branch since July, and the last release was over a year ago.

I have the same question. WeaveWorks has another project called footloose which allows docker containers to be used like VMs. Also dead. The only other project/company (aside from Kata) promoting firecracker was Talos. But not sure if they maintain their integration still. https://www.talos.dev/v1.2/reference/cli/#talosctl-cluster-c...

Talos no longer supports creating a cluster automatically in a firecracker VM (which you can still do in docker, via `talosctl cluster create`) but you can certainly run Talos Linux in firecracker VMs and create a cluster that way.

Re: Ignite – Use Firecracker VMs with Docker images

#78

Post by nailer »

NathanKP wrote: post #72
nailer wrote: post #54

Can someone help me understand whether this is: - My app per the Dockerfile - Container - Firecracker VM Or: - My app per the Dockerfile - Firecracker VM Thanks!

The latter. If you typed `docker ps` on your host there would be nothing showing up. The application is running inside of a micro VM. It should also be noted that containerized processes that are running on a host will still show up on the host's process tree if you look at the output of `ps`, `htop` or similar commands. Containers aren't real boundaries, just namespaces on the host machine. This is the not the case…

Thanks for the answer.

Oh then this is awesome. The convenience of Docker images with the proper isolation of MicroVMs.

Re: Ignite – Use Firecracker VMs with Docker images

#79

Post by solarkraft »

pjmlp wrote: post #68
sofixa wrote: post #34

Earlier quoted context omitted.

> Why not just make a base image and some simple setup scripts? That makes it possible for you to create an image (VM or container?, good start. But what Firecracker and containers solve is isolation and ease of distribution.

Just like VM repos used to be used....

Yup, but with good tooling.

Maybe I'm missing something, but all traditional approaches I've seen for VM management treat them as pets, not cattle.

Re: Ignite – Use Firecracker VMs with Docker images

#80

Post by Kalanos »

lbotos wrote: post #62
Kalanos wrote: post #61

can someone help me understand why VMs are necessary here?

"Isolation" right? By making that boundary, you get "slightly safer" code execution in a multi-tenant environment.

ah okay. kind of a brute force way to go about it. why bother with containers at that point? just use VM images
Post reply on HN