Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coding Agent VMs on NixOS with Microvm.nix

michael.stapelberg.ch

Post reply on HN
Print view View original

21–30 of 58 posts

Re: Coding Agent VMs on NixOS with Microvm.nix

#21

Post by phrotoma »

NJL3000 wrote: post #16

A pair of containers felt a bit cheaper than a VM: https://github.com/5L-Labs/amp_in_a_box I was going to add Gemini / OpenCode Kilo next. There is some upfront cost to define what endpoints to map inside, but it definitely adds a veneer of preventing the crazy…

One problem with using containers as an isolation environment for a coding assistant is that it becomes challenging to have the agent work on a containerized project. You often need some janky "docker-in-docker" nonsense that hampers efforts.

Re: Coding Agent VMs on NixOS with Microvm.nix

#23

Post by aquariusDue »

Cyph0n wrote: post #12
rootnod3 wrote: post #5

That is quite an involved setup to get a costly autocomplete going. Is that really where we are at? Just outsource convenience to a few big players that can afford the hardware? Just to save on typing and god forbid…thinking? “Sorry boss, I can’t write code because cloudflare is down.”

Keep in mind that this setup is a one-time cost. Also, a lot of the code is related to configuring it the way the author wants it (via Home Manager). Generally speaking, once you have a working NixOS config, incremental changes become extremely trivial, safe, and easy to rollback.

To provide another data point: I too use NixOS and oh boy that one-time is really costly. And while we're sharing Nix stuff for LLMs there's this piece of kit too: https://github.com/YPares/rigup.nix

Re: Coding Agent VMs on NixOS with Microvm.nix

#24

Post by alexzenla »

clawsyndicate wrote: post #2

we run ~10k agent pods on k3s and went with gvisor over microvms purely for density. the memory overhead of a dedicated kernel per tenant just doesn't scale when you're trying to pack thousands of instances onto a few nodes. strict network policies and pid limits cover most of the isolation gaps anyway.

This is a big reason for our strategy at Edera (https://edera.dev) of building hypervisor technology that eliminates the standard x86/ARM kernel overhead in favor of deep para-virtualization.

The performance of gVisor is often a big limiting factor in deployment.

Re: Coding Agent VMs on NixOS with Microvm.nix

#25

Post by 0x457 »

CuriouslyC wrote: post #14
rictic wrote: post #11

Earlier quoted context omitted.

Yep. What nix adds is a declarative and reproducible way to build customized OS images to boot into.

Nix is the best answer to "works on my machine," which is a problem I've seen at pretty much every place I've ever worked.

It's also an answer to caching with /nix/store. I wish more cloud services supported "give me your nixosConfiguration or something similar" instead of providing api to build containers/vms imperatively. Dockerfile and everything that mimics it is my least favorite way to do this.

Re: Coding Agent VMs on NixOS with Microvm.nix

#26

Post by alexzenla »

clawsyndicate wrote: post #4
secure wrote: post #3

Earlier quoted context omitted.

Yeah, when you run ≈10k agents instead of ≈10, you need a different solution :) I’m curious what gVisor is getting you in your setup — of course gVisor is good for running untrusted code, but would you say that gVisor prevents issues that would otherwise make the agent break out of the kubernetes pod? Like, do you have examples you’ve observed where gVisor has saved the day?

since we allow agents to execute arbitrary python, we treat every container as hostile. we've definitely seen logs of agents trying to crawl /proc or hit the k8s metadata api. gvisor intercepts those syscalls so they never actually reach the host kernel.

The reason why virtualization approaches with true Linux kernels is still important is what you do allow via syscalls ultimately does result in a syscall on the host system, even if through layers of indirection. Ultimately, if you fork() in gVisor, that calls fork() on the host (btw fork() execve() is expensive on gVisor still).

The middle ground we've built is that a real Linux kernel interfaces with your application in the VM (we call it a zone), but that kernel then can make specialized and specific interface calls to the host system.

For example with NVIDIA on gVisor, the ioctl()'s are passed through directly, with NVIDIA driver vulnerabilities that can cause memory corruption, it leads directly into corruption in the host kernel. With our platform at Edera (https://edera.dev), the NVIDIA driver runs in the VM itself, so a memory corruption bug doesn't percolate to other systems.

Re: Coding Agent VMs on NixOS with Microvm.nix

#27

Post by alexzenla »

souvik1997 wrote: post #20
clawsyndicate wrote: post #2

we run ~10k agent pods on k3s and went with gvisor over microvms purely for density. the memory overhead of a dedicated kernel per tenant just doesn't scale when you're trying to pack thousands of instances onto a few nodes. strict network policies and pid limits cover most of the isolation gaps anyway.

Hey @clawsyndicate I'd love to learn more about your use case. We are working on a product that would potentially get you the best of both worlds (microVM security and containers/gVisor scalability). My email is in my profile.

This is the thesis of our research paper here, a good middle ground is necessary: https://arxiv.org/abs/2501.04580

Re: Coding Agent VMs on NixOS with Microvm.nix

#28

Post by alexzenla »

rootnod3 wrote: post #9
clawsyndicate wrote: post #4

Earlier quoted context omitted.

since we allow agents to execute arbitrary python, we treat every container as hostile. we've definitely seen logs of agents trying to crawl /proc or hit the k8s metadata api. gvisor intercepts those syscalls so they never actually reach the host kernel.

And you see no problem in that at all? Just “throw a box around it and let the potentially malicious code run”? Wait until they find a hole. Then good luck.

This is why you can't build these microVM systems to just do isolation, it has to provide more value than that. Observability, policy, etc.

Re: Coding Agent VMs on NixOS with Microvm.nix

#29

Post by nh2 »

the_harpia_io wrote: post #6

[flagged]

I find that a bit of a weird point.

The goal of such sandboxing is that you can allow the agent to freely write/execute/test code during development, so that it can propose a solution/commit without the human having to approve every dangerous step ("write a Python file, then execute it" is already a dangerous step). As the post says: "To safely run a coding agent without review".

You would then review the code, and use it if it's good. Turning many small reviews where you need to be around and babysit every step into a single review at the end.

What you seem to be asking for (shipping the generated code to production without review) is a completely different goal and probably a bad idea.

If there really were a tool that can "scan the generated code" so reliably that it is safe to ship without human review, then that could just be part of the tool that generates the code in the first place so that no code scanning would be necessary. Sandboxing wouldn't be necessary either then. So then sandboxing wouldn't be "half the picture"; it would be unnecessary entirely, and your statement simplifies to "if we could auto-generate perfect code, we wouldn't need any of this".

Re: Coding Agent VMs on NixOS with Microvm.nix

#30

Post by NJL3000 »

phrotoma wrote: post #21
NJL3000 wrote: post #16

A pair of containers felt a bit cheaper than a VM: https://github.com/5L-Labs/amp_in_a_box I was going to add Gemini / OpenCode Kilo next. There is some upfront cost to define what endpoints to map inside, but it definitely adds a veneer of preventing the crazy…

One problem with using containers as an isolation environment for a coding assistant is that it becomes challenging to have the agent work on a containerized project. You often need some janky "docker-in-docker" nonsense that hampers efforts.

I was planning to have worktrees bind mounted systematically, but agree it’s not super clean atm at scale (yet)
Post reply on HN