We made our filesystem 47× faster by deleting it • Hacker News Board

Re: We made our filesystem 47× faster by deleting it

#41

Post by a_t48 » Sat, May 23, 2026, 11:46 PM UTC

appcypher wrote: post #29

a_t48 wrote: post #18

I've found that tar processing tends to dominate the time used to do anything with standard OCI layers. I have a more efficient format (that splits apart the layer into metadata+chunks) that I'm open sourcing soon if y'all are interested in using it.

interested. is the split for dedup, parallel pulls, or lazy loading specific files? maybe all. we've played with some chunking ideas on our end but haven't landed on a format. drop a link when it's out.

All of the above, plus being able to reflink to skip copies of large files, plus not having to round trip from disk a few times for tar layers, plus a number of other side benefits. Only using lazy loading for buildkit right now, as it does require FUSE and I want it to be opt in (for robotics contexts, for instance, you never want to lazy load).

Re: We made our filesystem 47× faster by deleting it

#42

Post by andix » Sun, May 24, 2026, 12:46 PM UTC

toast0 wrote: post #22

andix wrote: post #15

Isn't it really obvious that a user space fs will always be slow, and especially slow with small files? I don't know the purpose of microsandbox, but such an article doesn't give me great confidence in exploring it further.

> Isn't it really obvious that a user space fs will always be slow, and especially slow with small files? Small files seem like the perfect case for a user space fs... depending on what you mean by user space fs. If you mean interfacing with a FUSE (or similar) filesystem by using syscalls in your program to context switch into the kernel, then context switch to the userspace FUSE layer, then send it back to the kern…

If I got the article right, they were running FUSE inside the VM, and the VM's FUSE was talking to a process on the VM host (probably over virtual network). That can't be fast, not even theoretically.

Ever access to the fs had to go through two processes and two kernels, virtual networking, and probably even running on two different cores most of the time. That must be slow.

Re: We made our filesystem 47× faster by deleting it

#43

Post by toksdotdev » Sun, May 24, 2026, 10:12 PM UTC

cassianoleal wrote: post #40

toksdotdev wrote: post #30

Earlier quoted context omitted.

it depends where you want your agent to live. inside the sandbox, start a sandbox via the CLI and run your agents/do your dev in there. outside the sandbox, you'd configure your harness to use the MCP / skill integration. here's the guide: https://docs.microsandbox.dev/getting-started/agents if you're building a harness, the SDK provides better integration. let me know if you hit any blockers. for sync, it's currentl…

Awesome, thanks! If I run the agent outside and use the MCP, is it the model's responsibility to actually develop in the sandbox or are there deterministic guardrails against performing activities outside of it?

correct. when the sandbox is exposed via MCP, it's up to the model. the agent decides when to call it, and if it has any other way to run code (a shell tool, another MCP server), nothing forces it to go solely through the sandbox.

to actually enforce isolation, the sandbox needs to be the only way the agent can run code. either don't give it any other code-running tools, or the harness exposes sandboxing as a first-class concept. for the latter, anthropic makes this possible with self-hosted execution: https://platform.claude.com/docs/en/managed-agents/self-host...

happy to answer any other questions.