Live data from Hacker News

We made our filesystem 47× faster by deleting it

microsandbox.dev

31–40 of 43 posts

Re: We made our filesystem 47× faster by deleting it

#31

>Every file operation inside the VM had to bounce out to the host through FUSE Lol, yeah that was your mistake. FUSE is a phenomenal idea but anyone who has used it knows how slow it can be.

I have learned first hand with my agentic workflow as it took 1 hour to compile rust instead of 6 seconds.

I don't think you can blame that on FUSE in general. If not some quirk of your local setup then maybe the particular implementation you were using - what sort of volume was it?

Re: We made our filesystem 47× faster by deleting it

#32
post #20

Earlier quoted context omitted.

FUSE is not slow. Our distributed file system pipes over 70 GB/a through a single mount point.

70GB/s or 70 Gbps?

OP said 70GB/a so I'm gonna assume that's gigabytes per annum /j

Re: We made our filesystem 47× faster by deleting it

#33

Earlier quoted context omitted.

I have learned first hand with my agentic workflow as it took 1 hour to compile rust instead of 6 seconds.

I don't think you can blame that on FUSE in general. If not some quirk of your local setup then maybe the particular implementation you were using - what sort of volume was it?

it was dirty pages, kernel was artificially throttling it inside a sandbox.

Re: We made our filesystem 47× faster by deleting it

#35
post #15

Isn't it really obvious that a user space fs will always be slow, and especially slow with small files? I don't know the purpose of microsandbox, but such an article doesn't give me great confidence in exploring it further.

Not obvious to me.

A filesystem is a database, it is a type of key value storage. there is a hierarchical lookup key that points you to an unstructured block of data.

Many databases(berkleydb, postgresql, sqlite) are then built on this unstructured database. There is absolutely nothing indicating that putting putting a key value database with hierarchical keys and unstructured blocks in a single file will be slow.

It could be, naive indexing or rebalancing could be very slow. But it does not have to be. In fact berkleydb is a neat case study here. superficially it is a ridiculously simple key value store, why does such a simple thing even need to exist, or have such a long lived presence. It turns out building the efficient structures needed to work with slow non-volatile storage is non-trivial. Early mysql used berkleydb as a low level storage engine. Note that mysql main selling point was speed before correctness.

See also: Virtual machines another ubiquitous case of a filesystem in userspace.

Re: We made our filesystem 47× faster by deleting it

#37
post #35
post #15

Isn't it really obvious that a user space fs will always be slow, and especially slow with small files? I don't know the purpose of microsandbox, but such an article doesn't give me great confidence in exploring it further.

Not obvious to me. A filesystem is a database, it is a type of key value storage. there is a hierarchical lookup key that points you to an unstructured block of data. Many databases(berkleydb, postgresql, sqlite) are then built on this unstructured database. There is absolutely nothing indicating that putting putting a key value database with hierarchical keys and unstructured blocks in a single file will be slow. It…

It's something completely different. A database like SQLite runs in the same process as the application

All the filesystem calls go through the kernel. A userspace file system is another process.

It's not like SQLite, it's more like Postgres. Try sending a few hundred thousand small queries to Postgres, and be surprised how slow it's going to be.

The file system api is not like sql that allows complex queries. It's a lot of tiny and simple requests that assume very low latency.

Re: We made our filesystem 47× faster by deleting it

#38
post #28
post #15

Isn't it really obvious that a user space fs will always be slow, and especially slow with small files? I don't know the purpose of microsandbox, but such an article doesn't give me great confidence in exploring it further.

Sqlite is essentially a user space queryable file system and it can be faster than writing to file system directly while working with small files.

SQLite is in-process. A user space file system is another process. Like Postgres if we want to compare fs with dbs. And Postgres is slow for many small queries, like a userspace file system.

Re: We made our filesystem 47× faster by deleting it

#40

Is anyone here using this software? How do you integrate it with your agent workflow? Do you run agents in editor (Zed, VS Code, Cursor, whatever)? Have you tried the sync feature? Edit: FML why is this being downvoted? At least have the decency of explaining, I'm happy to adjust my conduct but I can't do so if I don't know what I did wrong.

it depends where you want your agent to live. inside the sandbox, start a sandbox via the CLI and run your agents/do your dev in there. outside the sandbox, you'd configure your harness to use the MCP / skill integration. here's the guide: https://docs.microsandbox.dev/getting-started/agents if you're building a harness, the SDK provides better integration. let me know if you hit any blockers. for sync, it's currentl…

Awesome, thanks!

If I run the agent outside and use the MCP, is it the model's responsibility to actually develop in the sandbox or are there deterministic guardrails against performing activities outside of it?

Post reply on HN