Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code and Let Live

fly.io

Post reply on HN
Print view View original

11–20 of 195 posts

Re: Code and Let Live

#11

Post by HumanOstrich »

skybrian wrote: post #3

This sounds great and it's roughly what exe.dev is doing too. Coincidence?

Not really. One of the primary features of sprites.dev that I don't see anywhere on exe.dev is a fast way to create and restore checkpoints, like a git repo for your entire VM.

This is needed for sandboxes if you don't want to throw them away and start over when something goes wrong.

With sprites.dev you can create an additional checkpoint and then turn Claude Code (or your preferred agent) loose to do anything. Even if it burns down the sandbox you can just restore a checkpoint in about a second.

Re: Code and Let Live

#12

Post by tptacek »

indigodaddy wrote: post #7

So this is neat and useful and I think will/should get traction. So let's say sprite is my building/dev ground floor. I get my thing/app to where I want it, but at the end of the day I think my thing/app is so awesome that it should be a production app for the whole world, and, I want to actually deploy it on fly, say. Have you guys thought about that workflow, and what it might take to push button/migrate a sprite a…

It depends on which Fly person you talk to. If you talk to Kurt he'll try to sell you on his crazy dream of how all software is going to be malleable and "prod" doesn't mean anything anymore. If you ask me: tell Claude to make a Dockerfile of the current state of your Sprite, and then deploy it as a Fly Machine. It's a good question, and we're working out how the transition from Sprite to Fly Machine works, but that's how I'd do it today.

I don't think we're going to do anything new with GPUs any time soon.

Re: Code and Let Live

#13

Post by dtkav »

fly.io is doing really good work. I've super enjoyed building our product on their platform. I love fly-replay combined with super fast start-up.

I've been thinking a lot about how to run agents (and skills) securely while giving them a lot of powerful capabilities.

I recently used their macaroons library to turn arbitrary API keys (e.g. for stripe's API) into macaroons. I route requests for an upstream host (like stripe) through Envoy as a mitm proxy which injects the real creds after verifying the macaroon.

It is such a powerful pattern. I'm always worried about leaking sensitive keys through prompt injection attacks (or just sending them to anthropic), but in this model you can attenuate the keys (both capabilities & validity window) client side. The Envoy proxy lives inside my flycast network so it can't be accessed externally.

It would be so cool if fly built something like this into sprites.dev (though I can see how it would be spooky to have fly install their own certs for stripe, etc...)

Re: Code and Let Live

#14

Post by tptacek »

dtkav wrote: post #13

fly.io is doing really good work. I've super enjoyed building our product on their platform. I love fly-replay combined with super fast start-up. I've been thinking a lot about how to run agents (and skills) securely while giving them a lot of powerful capabilities. I recently used their macaroons library to turn arbitrary API keys (e.g. for stripe's API) into macaroons. I route requests for an upstream host (like st…

If you read Ben Toews work on the tokenizer you have a good sense of where I want Sprites to go with key leaks and prompt injection:

https://fly.io/blog/tokenized-tokens/

Re: Code and Let Live

#15

Post by dtkav »

tptacek wrote: post #14
dtkav wrote: post #13

fly.io is doing really good work. I've super enjoyed building our product on their platform. I love fly-replay combined with super fast start-up. I've been thinking a lot about how to run agents (and skills) securely while giving them a lot of powerful capabilities. I recently used their macaroons library to turn arbitrary API keys (e.g. for stripe's API) into macaroons. I route requests for an upstream host (like st…

If you read Ben Toews work on the tokenizer you have a good sense of where I want Sprites to go with key leaks and prompt injection: https://fly.io/blog/tokenized-tokens/

Awesome stuff! Thanks for the reply.

Tokenizer is an explicit proxy though right?

My use case is very similar, but I wanted a transparent proxy so I could run unmodified scripts. It is a tricky design decision though.

I also mount a little fuse filesystem that mints macaroon on read (with a shorter lifetime, probably inspired by y'all but i forget from where).

I work on realtime collaboration of markdown files (currently in Obsidian), which has become a shared-context substrate for agents, skills, etc.. Our own company workspace has skills that have scoped access to fly, stripe, gmail, etc. We're definitely drinking the file-over-app personal-software-for-teams Kool-Aid, so the problem space for us includes access control and auditing.

Love your work :)

Re: Code and Let Live

#17

Post by tptacek »

nextaccountic wrote: post #16

How exactly can code agents make use of this? You install claude code inside a Sprite and run it there? Do you also need to put all your codebase in this sprite?

Claude Code is already in the Sprite; just create one and type "claude". But they have an API and Claude (or Gemini or Codex) can use them remotely too. They're disposable computers. Use them however you want.

Re: Code and Let Live

#18

Post by qhwudbebd »

AFAIK fly.io run firecracker and cloud-hypervisor VMs. This seems to have a copy-on-write filesystem underneath.

Given their principled take on only trusting full-VM boundaries, I doubt they moved any of the storage stack into the untrusted VM.

So maybe a virtio-block device passing through discard to some underlying CoW storage stack, or maybe virtio-fs if it's running on ch instead of fc? Would be interesting to hear more about the underlying design choices and trade-offs.

Edit: from their website, "Since it's just ext4, you won't run into weird edge cases like you might with NFS or FUSE mounts. You can happily use shared memory files, for example, so you can run SQLite in all its modes." So it's a virtio block device supporting discard that's exposed to the VM. Interesting; fc doesn't support virtio discard passthrough, and support for ch is still in progress...

Re: Code and Let Live

#19

Post by a_lanfranco »

sprites.dev looks very interesting to me. Is there a way to set up a limit to how much scaling a sprite can get, or to set a spending limit? I wouldn't want to spin something up, and then be surprised by an unexpectedly high bill.

Re: Code and Let Live

#20

Post by mwcampbell »

I want something like this, but running on my own box. I now have a Linux box with plenty of RAM and storage under my desk. (It happens to be an NVIDIA DGX Spark, but I'm not really interested in passing the GPU through to these sandboxed VMs; I know that's not practical anyway.) Maybe I'll see if I can hack together a local solution like this using Firecracker.
Post reply on HN