Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Sub-millisecond VM sandboxes using CoW memory forking

github.com

Post reply on HN
Print view View original

51–60 of 85 posts

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#51

Post by Intermernet »

cperciva wrote: post #30
Retr0id wrote: post #26

Earlier quoted context omitted.

I suppose it'd be easy enough to re-seed RNGs, but re-relocating ASLR sounds like a pain. (Although I suppose for Python that doesn't matter)

Re-seeding is easy. The hard parts are (a) finding everything which needs to be reseeded -- not just explicit RNGs but also things like keys used to pick outgoing port numbers in a pseudorandom order -- and (b) making sure that all the relevant code becomes aware that it was just forked -- not necessarily trivial given that there's no standard "you just got restarted from a snapshot" signal in UNIX.

I would have thought that in the days of containers, we'd have better tooling around this. Sounds like a goldmine for vuln research!

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#52

Post by Intermernet »

lionkor wrote: post #47
benterix wrote: post #46

Earlier quoted context omitted.

I don't mean to turn this into a religious war, but honestly, I sometimes wonder what would be the net benefit for humanity if Windows slowly disappeared. And I'm saying this as someone who appreciates the good stuff done by Microsoft in the past (windows 9* UI, decades-long support for Win32 APIs etc.).

Why doesn't Microsoft just take their incredible, human-replacing, AGI level AI's, and just port all their code to a Linux kernel instead of the NT kernel? Oh right, because that's not in the training set.

The NT kernel is actually pretty amazing. You can even run a pretty solid Windows version if you want to sail the high seas. LTSC and masgrave will get you most of the way there.

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#53

Post by user_7832 »

adammiribyan wrote: post #38
wei03288 wrote: post #37

[flagged]

On tail latency: KVM VM creation is 99.5% of the fork cost - create_vm, create_irq_chip, create_vcpu, and restoring CPU state. The CoW mmap is ~4 microseconds regardless of load. P99 at 1000 concurrent is 1.3ms. The mmap CoW page faults during execution are handled transparently by the host kernel and don't contribute to fork latency. On snapshot staleness: yes, forks inherit all internal state including RNG seeds. F…

Just a head's up, you're almost certainly replying to a bot. For some reason there's a ton of them in this post.

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#54

Post by BornaP »

injidup wrote: post #40

It's so frustrating seeing all this sandbox tooling pop up for linux but windows is soooooo far behind. I mean Windows Sandbox ( https://learn.microsoft.com/en-us/windows/security/applicati... ) doesn't even have customizable networking white lists. You can turn networking on or off but that's about as fine grained as it gets. So all of us still having to write desktop windows stuff are left without a good method of…

Feel you. That's why we're actively working on Windows and macOS sandbox support at Daytona - with proper isolation, agents tools, dynamic resizing etc; not just "networking on/off" level controls.

If you're building agents on Windows and want to give it a spin, reach out for early access.

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#55

Post by ivan_burazin »

injidup wrote: post #40

It's so frustrating seeing all this sandbox tooling pop up for linux but windows is soooooo far behind. I mean Windows Sandbox ( https://learn.microsoft.com/en-us/windows/security/applicati... ) doesn't even have customizable networking white lists. You can turn networking on or off but that's about as fine grained as it gets. So all of us still having to write desktop windows stuff are left without a good method of…

Can give you access to win sandboxes on Daytona, just fill this in here and lmk! https://www.daytona.io/docs/en/computer-use/

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#56

Post by BornaP »

Really impressive work. Sub-millisecond cold starts via CoW forking is a pretty clever approach.

The tricky part we keep running into with agent sandboxes is that code execution is just one piece, bcs agents also need file system access, memory, git, a pty, and a bunch of other tools all wired up and isolated together. That's where things get hairy fast.

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#57

Post by shayonj »

adammiribyan wrote: post #34
diptanu wrote: post #15

The tricky part of doing this in production is cloning sandboxes across nodes. You would have to snapshot the resident memory, file system (or a CoW layer on top of the rootfs), move the data across nodes, etc.

Agreed, cross-node is the hard next step. For now single-node density gets you surprisingly far. 1000 concurrent sandboxes on one $50 box. When we need multi-node, userfaultfd with remote page fetch is the likely path.

Cool project. +1 on userfaultfd for the multi-node path. Wrote about how uffd-based on-demand restore works wrt to my Cloud Hypervisor change [1] if you are curious.

I think the the main things to watch are fault storms at resume (all vCPUs hitting missing pages at once) and handler throughput if you're serving pages over the network instead of local mmap. I think its less likely to happen when you fork a brand new VM vs say a VM that has been doing things for 5 mins.

Also interestingly, Cloud Hypervisor couldn't use MAP_PRIVATE for this because it breaks VFIO/vhost-user bindings. Firecracker's simpler device model is nice for cases like this.

[1] https://www.shayon.dev/post/2026/65/linux-page-faults-mmap-a...

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#58

Post by aa-jv »

Nice.

Now I want the ability to freeze the VM cryogenically and move it to another machine automagically, defrosting and running as seamlessly as possible.

I know this is gonna happen soon enough, I've been waiting since the death of TandemOS for just this feature ..

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#59

Post by aa-jv »

cperciva wrote: post #30
Retr0id wrote: post #26

Earlier quoted context omitted.

I suppose it'd be easy enough to re-seed RNGs, but re-relocating ASLR sounds like a pain. (Although I suppose for Python that doesn't matter)

Re-seeding is easy. The hard parts are (a) finding everything which needs to be reseeded -- not just explicit RNGs but also things like keys used to pick outgoing port numbers in a pseudorandom order -- and (b) making sure that all the relevant code becomes aware that it was just forked -- not necessarily trivial given that there's no standard "you just got restarted from a snapshot" signal in UNIX.

Isn't this what -HUP is supposed to be for in the first place? Maybe a -STOP/-HUP/-HUP situation?

Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking

#60

Post by shayonj »

adammiribyan wrote: post #3
skwuwu wrote: post #2

I noticed that you implemented a high-performance VM fork. However, to me, it seems like a general-purpose KVM project. Is there a reason why you say it is specialized for running AI agents?

Fair question. The fork engine itself is general purpose -- you could use it for anything that needs fast isolated execution. We say 'AI agents' because that's where the demand is right now. Every agent framework (LangChain, CrewAI, OpenAI Assistants) needs sandboxed code execution as a tool call, and the existing options (E2B, Daytona, Modal) all boot or restore a VM/container per execution. At sub-millisecond fork…

> you can do things that aren't practical with 100-200ms startup: speculative parallel execution (fork 10 VMs, try 10 approaches, keep the best), treating code execution like a function call instead of an infrastructure decision, etc.

i am not following, why isn't it practical?

Post reply on HN