[dead]
Show HN: Sub-millisecond VM sandboxes using CoW memory forking
71–80 of 85 posts
Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking
#72Earlier quoted context omitted.
Re-seeding is easy. The hard parts are (a) finding everything which needs to be reseeded -- not just explicit RNGs but also things like keys used to pick outgoing port numbers in a pseudorandom order -- and (b) making sure that all the relevant code becomes aware that it was just forked -- not necessarily trivial given that there's no standard "you just got restarted from a snapshot" signal in UNIX.
Isn't this what -HUP is supposed to be for in the first place? Maybe a -STOP/-HUP/-HUP situation?
HUP is short for "hangup" which was supposed to be sent when the tty controlling the session the process is in hung up.
Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking
#73Any plans to offer self-hosted / open-source version?
Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking
#74Earlier quoted context omitted.
Feel you. That's why we're actively working on Windows and macOS sandbox support at Daytona - with proper isolation, agents tools, dynamic resizing etc; not just "networking on/off" level controls. If you're building agents on Windows and want to give it a spin, reach out for early access.
Our stack is msvc / cmake / ninja / incredibuild ? Can you support such things?
We should be able to. The goal is that if it runs on Windows, it runs in the sandbox. That's the whole point.
fill this out - https://docs.google.com/forms/d/e/1FAIpQLSfoK-77-VpfsMubw8F4...
Put under use case HN thread and i'll ping you back
Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking
#75Is this a service or a library? The README has curl and an API key... Can I run this myself on my own hardware?
Both. The engine is open source. You can self-host it on any Linux box with KVM. There's also a live API you can hit right now (curl example in the README). Building the managed service for teams that don't want to run their own infra.
Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking
#76Earlier quoted context omitted.
Agreed, cross-node is the hard next step. For now single-node density gets you surprisingly far. 1000 concurrent sandboxes on one $50 box. When we need multi-node, userfaultfd with remote page fetch is the likely path.
Cool project. +1 on userfaultfd for the multi-node path. Wrote about how uffd-based on-demand restore works wrt to my Cloud Hypervisor change [1] if you are curious. I think the the main things to watch are fault storms at resume (all vCPUs hitting missing pages at once) and handler throughput if you're serving pages over the network instead of local mmap. I think its less likely to happen when you fork a brand new V…
Great writeup, bookmarked. The fault storm point is interesting -- our forks are short-lived (execute and discard) so the working set is small, but for longer-running sandboxes that would absolutely be a problem.
Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking
#77Your write-up made me think of: https://codesandbox.io/blog/how-we-clone-a-running-vm-in-2-s... Are there parallels?
I think this is very similar! Really cool to see. The first version we launched used the exact same approach (MAP_PRIVATE). Later on, we bypassed the file system by using shared memory and using userfaultfd because ultimately the NVMe became the bottleneck ( https://codesandbox.io/blog/cloning-microvms-using-userfault... and https://codesandbox.io/blog/how-we-scale-our-microvm-infrast... ).
Glad to see the approach validated at scale! I hadn't seen your blog posts until they were linked here, going to dig into the userfaultfd path. Would love to chat if you're open to it.
Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking
#78It's so frustrating seeing all this sandbox tooling pop up for linux but windows is soooooo far behind. I mean Windows Sandbox ( https://learn.microsoft.com/en-us/windows/security/applicati... ) doesn't even have customizable networking white lists. You can turn networking on or off but that's about as fine grained as it gets. So all of us still having to write desktop windows stuff are left without a good method of…
You can run WSL on Windows — then you've got access to all the Linux sandbox tools.
Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking
#79Really impressive work. Sub-millisecond cold starts via CoW forking is a pretty clever approach. The tricky part we keep running into with agent sandboxes is that code execution is just one piece, bcs agents also need file system access, memory, git, a pty, and a bunch of other tools all wired up and isolated together. That's where things get hairy fast.
[dead]
Re: Show HN: Sub-millisecond VM sandboxes using CoW memory forking
#80[dead]