Coding Agent VMs on NixOS with Microvm.nix
michael.stapelberg.ch
Coding Agent VMs on NixOS with Microvm.nix
1–10 of 58 posts
Re: Coding Agent VMs on NixOS with Microvm.nix
#2we run ~10k agent pods on k3s and went with gvisor over microvms purely for density. the memory overhead of a dedicated kernel per tenant just doesn't scale when you're trying to pack thousands of instances onto a few nodes. strict network policies and pid limits cover most of the isolation gaps anyway.
Re: Coding Agent VMs on NixOS with Microvm.nix
#3we run ~10k agent pods on k3s and went with gvisor over microvms purely for density. the memory overhead of a dedicated kernel per tenant just doesn't scale when you're trying to pack thousands of instances onto a few nodes. strict network policies and pid limits cover most of the isolation gaps anyway.
Yeah, when you run ≈10k agents instead of ≈10, you need a different solution :)
I’m curious what gVisor is getting you in your setup — of course gVisor is good for running untrusted code, but would you say that gVisor prevents issues that would otherwise make the agent break out of the kubernetes pod? Like, do you have examples you’ve observed where gVisor has saved the day?
Re: Coding Agent VMs on NixOS with Microvm.nix
#4we run ~10k agent pods on k3s and went with gvisor over microvms purely for density. the memory overhead of a dedicated kernel per tenant just doesn't scale when you're trying to pack thousands of instances onto a few nodes. strict network policies and pid limits cover most of the isolation gaps anyway.
Yeah, when you run ≈10k agents instead of ≈10, you need a different solution :) I’m curious what gVisor is getting you in your setup — of course gVisor is good for running untrusted code, but would you say that gVisor prevents issues that would otherwise make the agent break out of the kubernetes pod? Like, do you have examples you’ve observed where gVisor has saved the day?
since we allow agents to execute arbitrary python, we treat every container as hostile. we've definitely seen logs of agents trying to crawl /proc or hit the k8s metadata api. gvisor intercepts those syscalls so they never actually reach the host kernel.
Re: Coding Agent VMs on NixOS with Microvm.nix
#5That is quite an involved setup to get a costly autocomplete going.
Is that really where we are at? Just outsource convenience to a few big players that can afford the hardware? Just to save on typing and god forbid…thinking?
“Sorry boss, I can’t write code because cloudflare is down.”
Re: Coding Agent VMs on NixOS with Microvm.nix
#6[flagged]
Re: Coding Agent VMs on NixOS with Microvm.nix
#7we run ~10k agent pods on k3s and went with gvisor over microvms purely for density. the memory overhead of a dedicated kernel per tenant just doesn't scale when you're trying to pack thousands of instances onto a few nodes. strict network policies and pid limits cover most of the isolation gaps anyway.
Yeah, when you run ≈10k agents instead of ≈10, you need a different solution :) I’m curious what gVisor is getting you in your setup — of course gVisor is good for running untrusted code, but would you say that gVisor prevents issues that would otherwise make the agent break out of the kubernetes pod? Like, do you have examples you’ve observed where gVisor has saved the day?
I've used both gVisor and microvms for this (at very large scales), and there are various tradeoffs between the two.
The huge gVisor drawback is that it __drastically_ slows down applications (despite startup time being faster.)
For agents, the startup time latency is less of an issue than the runtime cost, so microvms perform a lot better. If you're doing this in kube, then there's a bunch of other challenges to deal with if you want standard k8s features, but if you're just looking for isolated sandboxes for agents, microvms work really well.
Re: Coding Agent VMs on NixOS with Microvm.nix
#8Couldn't you replicate all of your setup with qemu microvm?
Without nix I mean
Re: Coding Agent VMs on NixOS with Microvm.nix
#9Earlier quoted context omitted.
Yeah, when you run ≈10k agents instead of ≈10, you need a different solution :) I’m curious what gVisor is getting you in your setup — of course gVisor is good for running untrusted code, but would you say that gVisor prevents issues that would otherwise make the agent break out of the kubernetes pod? Like, do you have examples you’ve observed where gVisor has saved the day?
since we allow agents to execute arbitrary python, we treat every container as hostile. we've definitely seen logs of agents trying to crawl /proc or hit the k8s metadata api. gvisor intercepts those syscalls so they never actually reach the host kernel.
And you see no problem in that at all? Just “throw a box around it and let the potentially malicious code run”?
Wait until they find a hole. Then good luck.
Re: Coding Agent VMs on NixOS with Microvm.nix
#10we run ~10k agent pods on k3s and went with gvisor over microvms purely for density. the memory overhead of a dedicated kernel per tenant just doesn't scale when you're trying to pack thousands of instances onto a few nodes. strict network policies and pid limits cover most of the isolation gaps anyway.
LXC containers inside a VM scales. bonus point that LXC containers feel like a VM.