Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Sub-millisecond VM sandboxes using CoW memory forking

github.com

Post reply on HN
Print view View original

1–10 of 85 posts

Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#1

Post by adammiribyan »

I wanted to see how fast an isolated code sandbox could start if I never had to boot a fresh VM.

So instead of launching a new microVM per execution, I boot Firecracker once with Python and numpy already loaded, then snapshot the full VM state. Every execution after that creates a new KVM VM backed by a `MAP_PRIVATE` mapping of the snapshot memory, so Linux gives me copy-on-write pages automatically.

That means each sandbox starts from an already-running Python process inside a real VM, runs the code, and exits.

These are real KVM VMs, not containers: separate guest kernel, separate guest memory, separate page tables. When a VM writes to memory, it gets a private copy of that page.

The hard part was not CoW itself. The hard part was resuming the snapshotted VM correctly.

Rust, Apache 2.0.

Show HN: Sub-millisecond VM sandboxes using CoW memory forking
github.com

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#3

Post by adammiribyan »

skwuwu wrote: post #2

I noticed that you implemented a high-performance VM fork. However, to me, it seems like a general-purpose KVM project. Is there a reason why you say it is specialized for running AI agents?

Fair question. The fork engine itself is general purpose -- you could use it for anything that needs fast isolated execution. We say 'AI agents' because that's where the demand is right now. Every agent framework (LangChain, CrewAI, OpenAI Assistants) needs sandboxed code execution as a tool call, and the existing options (E2B, Daytona, Modal) all boot or restore a VM/container per execution. At sub-millisecond fork times, you can do things that aren't practical with 100-200ms startup: speculative parallel execution (fork 10 VMs, try 10 approaches, keep the best), treating code execution like a function call instead of an infrastructure decision, etc.

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#4

Post by jauntywundrkind »

I keep so so so many opencode windows going. I wish I had bought a better SSD, because I have so much swap space to support it all.

I keep thinking I need to see if CRIU (checkpoint restore in userspace) is going to work here. So I can put work down for longer time, be able to close & restore instances sort of on-demand.

I don't really love the idea of using VMs more, but I super love this project. Heck yes forking our processes/VMs.

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#7

Post by adammiribyan »

yagizdagabak wrote: post #5

Cool approach. Are you guys planning on creating a managed version?

The API in the readme is live right now -- you can curl it. Plan is multi-region, custom templates with your own dependencies, and usage-based pricing. Email in my profile if you want early access.

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#8

Post by adammiribyan »

jauntywundrkind wrote: post #4

I keep so so so many opencode windows going. I wish I had bought a better SSD, because I have so much swap space to support it all. I keep thinking I need to see if CRIU (checkpoint restore in userspace) is going to work here. So I can put work down for longer time, be able to close & restore instances sort of on-demand. I don't really love the idea of using VMs more, but I super love this project. Heck yes forking o…

CRIU is great for save/restore. The nice thing about CoW forking is it's cheap branching, not just checkpointing. You can clone a running state thousands of times at a few hundred KB each.
Post reply on HN