Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coding Agent VMs on NixOS with Microvm.nix

michael.stapelberg.ch

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Coding Agent VMs on NixOS with Microvm.nix

#31

Post by souvik1997 »

alexzenla wrote: post #24
clawsyndicate wrote: post #2

we run ~10k agent pods on k3s and went with gvisor over microvms purely for density. the memory overhead of a dedicated kernel per tenant just doesn't scale when you're trying to pack thousands of instances onto a few nodes. strict network policies and pid limits cover most of the isolation gaps anyway.

This is a big reason for our strategy at Edera ( https://edera.dev ) of building hypervisor technology that eliminates the standard x86/ARM kernel overhead in favor of deep para-virtualization. The performance of gVisor is often a big limiting factor in deployment.

Edera looks very cool! Awesome team too.

I read the thesis on arxiv. Do you see any limitations from using Xen instead of KVM? I think that was the biggest surprise for me as I have very rarely seen teams build on Xen.

Re: Coding Agent VMs on NixOS with Microvm.nix

#32

Post by mystifyingpoi »

the_harpia_io wrote: post #6

[flagged]

> not just contain the execution environment.

See, my typical execution environment is a Linux vm or laptop, with a wide variety of SSH and AWS keys configured and ready to be stolen (even if they are temporary, it's enough to infiltrate prod, or do some sneaky lateral movement attack). On the other hand, typical application execution environment is an IAM user/role with strictly scoped permissions.

Re: Coding Agent VMs on NixOS with Microvm.nix

#33

Post by giancarlostoro »

nh2 wrote: post #29
the_harpia_io wrote: post #6

[flagged]

I find that a bit of a weird point. The goal of such sandboxing is that you can allow the agent to freely write/execute/test code during development, so that it can propose a solution/commit without the human having to approve every dangerous step ("write a Python file, then execute it" is already a dangerous step). As the post says: "To safely run a coding agent without review". You would then review the code, and u…

If that's the goal, why not just have Claude Code do it all from your phone at that point? Test it when its done locally you pull down the branch. Not 100% frictionless, but if it messes up an OS it would be anthropic's not yours.

Re: Coding Agent VMs on NixOS with Microvm.nix

#35

Post by giancarlostoro »

This brings me back to my college days. We had Windows, and Deep Freeze. Students could do anything on the computer, we restart it and its all wiped and new. How long before Deep Freeze realizes they could sell their tool to Vibe Coders, they have Deep Freeze for Mac but not for Linux, funnily enough.

Re: Coding Agent VMs on NixOS with Microvm.nix

#36

Post by Cyph0n »

aquariusDue wrote: post #23
Cyph0n wrote: post #12

Earlier quoted context omitted.

Keep in mind that this setup is a one-time cost. Also, a lot of the code is related to configuring it the way the author wants it (via Home Manager). Generally speaking, once you have a working NixOS config, incremental changes become extremely trivial, safe, and easy to rollback.

To provide another data point: I too use NixOS and oh boy that one-time is really costly. And while we're sharing Nix stuff for LLMs there's this piece of kit too: https://github.com/YPares/rigup.nix

Agreed, the learning curve is insane and docs are sparse. But it is truly worth it imo, even if you’re just using Nix as a build tool, or using Home Manager on Linux or macOS.

Re: Coding Agent VMs on NixOS with Microvm.nix

#37

Post by Cyph0n »

0x457 wrote: post #25
CuriouslyC wrote: post #14

Earlier quoted context omitted.

Nix is the best answer to "works on my machine," which is a problem I've seen at pretty much every place I've ever worked.

It's also an answer to caching with /nix/store. I wish more cloud services supported "give me your nixosConfiguration or something similar" instead of providing api to build containers/vms imperatively. Dockerfile and everything that mimics it is my least favorite way to do this.

It’s fairly trivial to map your NixOS config into a VM image: https://nixos.org/manual/nixos/stable/#sec-image-nixos-rebui...

An alternative is to “infect” a VM running in whatever cloud and convert it into a NixOS VM in-place: https://github.com/nix-community/nixos-anywhere

In fact, it is a common practice to use the latter to install NixOS on new machines. You start off by booting into a live USB with SSH enabled, then use nixos-anywhere to install NixOS and partition disks via disko. Here is an example I used recently to provision a new gaming desktop:

    nix run github:nix-community/nixos-anywhere -- \
      --flake .#myhost \
      --target-host user@192.168.0.100 \
      --generate-hardware-config nixos-generate-config ./hosts/myhost/hardware-configuration.nix
At the end of this invocation, you end up with a NixOS machine running your config partitioned based on your disk config. My disko config in this case (ZFS pool with 1 disk vdev): https://gist.github.com/aksiksi/7fed39f17037e9ae82c043457ed2...

Re: Coding Agent VMs on NixOS with Microvm.nix

#38

Post by pillbitsHQ »

Cyph0n wrote: post #37
0x457 wrote: post #25

Earlier quoted context omitted.

It's also an answer to caching with /nix/store. I wish more cloud services supported "give me your nixosConfiguration or something similar" instead of providing api to build containers/vms imperatively. Dockerfile and everything that mimics it is my least favorite way to do this.

It’s fairly trivial to map your NixOS config into a VM image: https://nixos.org/manual/nixos/stable/#sec-image-nixos-rebui... An alternative is to “infect” a VM running in whatever cloud and convert it into a NixOS VM in-place: https://github.com/nix-community/nixos-anywhere In fact, it is a common practice to use the latter to install NixOS on new machines. You start off by booting into a live USB with SSH enabled,…

[dead]

Re: Coding Agent VMs on NixOS with Microvm.nix

#39

Post by ATechGuy »

ryanrasti wrote: post #17
the_harpia_io wrote: post #6

[flagged]

Precisely! There's a fundamental tension: 1. Agents need to interact with the outside world to be useful 2. Interacting with the outside world is dangerous Sandboxes provide a "default-deny policy" which is the right starting point. But, current tools lack the right primitives to make fine grained data-access and data policy a reality. Object-capabilities provide the primitive for fine-grained access. IFC (informatio…

I agree. However, how to define these permissions when agent behavior is undefined?

Re: Coding Agent VMs on NixOS with Microvm.nix

#40

Post by groby_b »

rootnod3 wrote: post #5

That is quite an involved setup to get a costly autocomplete going. Is that really where we are at? Just outsource convenience to a few big players that can afford the hardware? Just to save on typing and god forbid…thinking? “Sorry boss, I can’t write code because cloudflare is down.”

If you believe "costly autocomplete" is all you get, you absolutely shouldn't bother.

You're opting for "sorry boss, it's going to take me 10 times as long, but it's going to be loving craftsmanship, not industrial production" instead. You want different tools, for a different job.

Post reply on HN