Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running MicroVMs in Proxmox VE, the Easy Way

taoofmac.com

Post reply on HN
Print view View original

11–20 of 52 posts

Re: Running MicroVMs in Proxmox VE, the Easy Way

#11

Post by traceroute66 »

alexellisuk wrote: post #8

This is clever work, especially given that Proxmox is already a very viable VMware replacement and wasn’t originally designed around microVMs as the primary abstraction. I’m glad this is working well for you. We’ve been on a similar journey, but came at it from the opposite direction. We started SlicerVM in 2022 after seeing how slow Multipass felt when launching more than one Linux VM, even though it is relatively l…

> We started SlicerVM ....

Shame you did not mention once in your long post that you are based on Firecracker, because I'm sure I'm not the first who was about to post "why is this better than Firecracker".

Also it is a shame you've adopted the subscription billing model instead of allowing people to buy perpetual licenses.

I dislike the subscription model in a pure sense, but also I dislike the "but its 'only' $x a month" argument oft-used by developers. Sure, in theory that's the case. But like everyone else in the world, I also have $x a month of other monthly expenses in my life, and I simply do not need or want N+1 software subscriptions. It all adds up.

The same applies to business environments, except the cost becomes even more exponential because you have (X-employees * N-subscriptions)/month.

Re: Running MicroVMs in Proxmox VE, the Easy Way

#12

Post by tobwen »

Thanks for the write-up, I like the integration in the Proxmox VE environment.

Given some similarities, I’d like to briefly mention `krun` here. Although it’s an OCI-compatible container runtime, it uses MicroVMs with a similar approach. Perhaps we can exchange ideas here? I recall that GPU passthrough is also a recurring topic there.

https://github.com/containers/crun/blob/main/krun.1.md

Re: Running MicroVMs in Proxmox VE, the Easy Way

#13

Post by nullify88 »

mkesper wrote: post #4

Tangentially, in theory, k3s + kubevirt + microvms sounds like the optimal combination for lightweight but isolated deployment. Does anyone have experience with that?

I think you might be looking for Kata Containers which is a CRI for running vmms like firecracker.

Re: Running MicroVMs in Proxmox VE, the Easy Way

#14

Post by sureglymop »

tobwen wrote: post #12

Thanks for the write-up, I like the integration in the Proxmox VE environment. Given some similarities, I’d like to briefly mention `krun` here. Although it’s an OCI-compatible container runtime, it uses MicroVMs with a similar approach. Perhaps we can exchange ideas here? I recall that GPU passthrough is also a recurring topic there. https://github.com/containers/crun/blob/main/krun.1.md

Krun is neat! I use it as podman backend. What I'm missing though is a good writeup on how to use it to sandbox as safely as possible. Already kind of difficult to know best with podman due to the sheer number of command line options and possible customizations.

I'm also a bit confused on how to use libkrun. It seems to be implemented in rust but provide a C API. Can it be used in rust projects?

Also, it made me curious if it would be possible to create a Linux distribution where every process runs in a microvm.

Re: Running MicroVMs in Proxmox VE, the Easy Way

#16

Post by crazysim »

tlamponi wrote: post #5

FWWI, we did evaluate and benchmark microVMs back in 2020. Back then it was not really seen worth it the maintenance cost compared to what it brought to the table, but it makes sense to re-evaluate that again soonish; with native dynamic load balancing and affinity rules (and further orchestration improvements being lined up) they might be better leveraged today. Oh, and mailing lists are a bliss to use compared to (…

I gotta ask but is Forgejo a barely loading forge? GitHub, GitLab being a pig sure but Forgejo seem pretty snappy.

Re: Running MicroVMs in Proxmox VE, the Easy Way

#17

Post by sorenjan »

Am I understanding the ballooning part right that this doesn't allocate all of the VM memory from the host until it's needed, and releases memory automatically when it's not needed anymore? So you can overprovision memory with multiple guests as long as the guests aren't using the memory at the same time?

Re: Running MicroVMs in Proxmox VE, the Easy Way

#18

Post by traverseda »

sorenjan wrote: post #17

Am I understanding the ballooning part right that this doesn't allocate all of the VM memory from the host until it's needed, and releases memory automatically when it's not needed anymore? So you can overprovision memory with multiple guests as long as the guests aren't using the memory at the same time?

Yes, it requires a daemon running inside the VM and can be finicky though.

Re: Running MicroVMs in Proxmox VE, the Easy Way

#19

Post by apitman »

My favorite microVM discovery recently has been tools built around libkrun. See smolvm[0].

The killer feature still missing from microVMs for me is the ability to enable CUDA support without passing through the entire GPU. vfio is just too much of a pain and too limiting. Sometimes I want to use my GPU on the host. Vulkan works fairly well with virtio-gpu and Venus, but I need CUDA. Venus is also still missing some important things like accelerated video encoding.

[0]: https://smolmachines.com/

Post reply on HN