Live data from Hacker News

Running MicroVMs in Proxmox VE, the Easy Way

taoofmac.com

21–30 of 52 posts

Re: Running MicroVMs in Proxmox VE, the Easy Way

#21
proxmox has been great although it comes with a learning curve

back when I used to use cursor I build this mcp but it should work for codex or claude

it lets me easily spin up vms with specs

its tough to create boxes now due to ram prices but got mine at a great time when it was very cheap; i just wish i had bought more then

https://github.com/agentify-sh/cursor-proxmox-mcp

Re: Running MicroVMs in Proxmox VE, the Easy Way

#22
post #5

FWWI, we did evaluate and benchmark microVMs back in 2020. Back then it was not really seen worth it the maintenance cost compared to what it brought to the table, but it makes sense to re-evaluate that again soonish; with native dynamic load balancing and affinity rules (and further orchestration improvements being lined up) they might be better leveraged today. Oh, and mailing lists are a bliss to use compared to (…

Yeah, I love this from a geek appeal and I have a beefy home lab (Dual Xeon Silver 4116 with 384GB, and 12TB of RAID10 SSD connected via 10gig), but being a homelab I want to eke out all my performance so I keep looking at LXC on my Proxmox box versus the optimization of VMs (does this machine really need 8GB or 6? Or 5?).

But when there's the discussion of the amount of time Qemu spends "in grub" and "probing legacy devices", maybe my use case is different, but my VMs aren't constantly being rebooted and when the VM is up it is near native speed so...

Re: Running MicroVMs in Proxmox VE, the Easy Way

#23

proxmox has been great although it comes with a learning curve back when I used to use cursor I build this mcp but it should work for codex or claude it lets me easily spin up vms with specs its tough to create boxes now due to ram prices but got mine at a great time when it was very cheap; i just wish i had bought more then https://github.com/agentify-sh/cursor-proxmox-mcp

Can you share more about the learning curve specifics you're referring to? Thanks.

Re: Running MicroVMs in Proxmox VE, the Easy Way

#24

This is clever work, especially given that Proxmox is already a very viable VMware replacement and wasn’t originally designed around microVMs as the primary abstraction. I’m glad this is working well for you. We’ve been on a similar journey, but came at it from the opposite direction. We started SlicerVM in 2022 after seeing how slow Multipass felt when launching more than one Linux VM, even though it is relatively l…

> We started SlicerVM .... Shame you did not mention once in your long post that you are based on Firecracker, because I'm sure I'm not the first who was about to post "why is this better than Firecracker". Also it is a shame you've adopted the subscription billing model instead of allowing people to buy perpetual licenses. I dislike the subscription model in a pure sense, but also I dislike the "but its 'only' $x a…

Author of a free open source alternative to slicer, based on a fork of libkrun (not firecracker based) that runs locally across wsl, macOS, and Linux natively.

https://github.com/smol-machines/smolvm

Re: Running MicroVMs in Proxmox VE, the Easy Way

#25
I'm curious if there's more than one way to achieve this.

Creating a single VM, with vm within vm (performance hit would be negligible for the orchestration work of agents), and it might offer some alternatives without having to customize Proxmox as much?

Re: Running MicroVMs in Proxmox VE, the Easy Way

#26
post #25

I'm curious if there's more than one way to achieve this. Creating a single VM, with vm within vm (performance hit would be negligible for the orchestration work of agents), and it might offer some alternatives without having to customize Proxmox as much?

This is what I do for a few projects. One beefy (Debian) VM with the CPU type set to "host" (or something like that) to allow for nested virtualization. Running Firecracker inside that VM.

Just be careful with the virtualized file systems to not create write amplification issues.

Re: Running MicroVMs in Proxmox VE, the Easy Way

#27
Very interesting work on microvms there. I would add that removing any kind of storage or file-system interaction is reachable for even faster bringup and removing the risk of attacks needing some form of persistence.

Also replacing network access with af-vsock is actually interesting if you want to simplify bring-up. SSH does some magic with vsocks these days too.

Re: Running MicroVMs in Proxmox VE, the Easy Way

#28
post #20

< 300ms boot with HW isolation sounds very nice but the pve-daemon patching approach is risky and might break at every eye-blink.

it's the correct way to do it though.

anyway, the author posted the sources on github and got in touch with the proxmox people, maybe they want to absorb that into the product (which would be very very cool).

Re: Running MicroVMs in Proxmox VE, the Easy Way

#29

This is clever work, especially given that Proxmox is already a very viable VMware replacement and wasn’t originally designed around microVMs as the primary abstraction. I’m glad this is working well for you. We’ve been on a similar journey, but came at it from the opposite direction. We started SlicerVM in 2022 after seeing how slow Multipass felt when launching more than one Linux VM, even though it is relatively l…

> We started SlicerVM .... Shame you did not mention once in your long post that you are based on Firecracker, because I'm sure I'm not the first who was about to post "why is this better than Firecracker". Also it is a shame you've adopted the subscription billing model instead of allowing people to buy perpetual licenses. I dislike the subscription model in a pure sense, but also I dislike the "but its 'only' $x a…

Yeah. I agree, I saw the SaaS-style pricing for running on my own infra and couldn't see any reason why I'd want this. I also don't see the technical upside to SlicerVM. It feels very risky given I've never heard of anyone actually running this in production. I think I'd take my chances with Proxmox plus microVM add-ons first.

Re: Running MicroVMs in Proxmox VE, the Easy Way

#30
post #19

My favorite microVM discovery recently has been tools built around libkrun. See smolvm[0]. The killer feature still missing from microVMs for me is the ability to enable CUDA support without passing through the entire GPU. vfio is just too much of a pain and too limiting. Sometimes I want to use my GPU on the host. Vulkan works fairly well with virtio-gpu and Venus, but I need CUDA. Venus is also still missing some i…

author of smolmachines here,

CUDA support is possible based on a cursory dive. I'll keep you posted on it

ref: https://thevirtualhorizon.com/2024/05/31/how-to-configure-th...

Post reply on HN