Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody likes lag: How to make low-latency dev sandboxes

compyle.ai

Post reply on HN
Print view View original

21–30 of 47 posts

Re: Nobody likes lag: How to make low-latency dev sandboxes

#21

Post by formerly_proven »

mlhpdx wrote: post #18
mnazzaro wrote: post #15

Earlier quoted context omitted.

While I can see that working well for echoing keystrokes in a terminal, I'm not sure how it would work when you actually enter commands into the terminal. Same for opening files in the IDE.

I didn’t get that the IDE is running on both sides, if that’s true. Wow.

This is why most IDEs nowadays ask you something about "trusting files" when opening a project. They tend to lick and run on everything in there (at least for dynamic-ish languages, and maybe not "run" intentionally but do stuff which is arbitrary code execution more or less by definition) to analyze the code.

Re: Nobody likes lag: How to make low-latency dev sandboxes

#22

Post by reactordev »

cedws wrote: post #19
iterateoften wrote: post #17

Why is there all the sudden an explosion of sandbox related posts and tools? Llms and agents always needed sandboxes… was it just the collective conscious decided all at once that it mattered and the area to focus building tools?

Particularly an explosion of SaaS sandboxes... why should I pay a subscription for some remote sandbox with paltry compute power, which I need a constant internet connection to access? I have this brilliant processor in my own laptop I want to use that I have already paid for, I don't want to use someone else's!

Some companies only allow access through a VDI like Windows Remote Desktop or some VMWare setup. It’s crazy.

Re: Nobody likes lag: How to make low-latency dev sandboxes

#24

Post by simonw »

iterateoften wrote: post #17

Why is there all the sudden an explosion of sandbox related posts and tools? Llms and agents always needed sandboxes… was it just the collective conscious decided all at once that it mattered and the area to focus building tools?

I think sandboxes are having their moment because it's become undeniable that coding agents are useful, and that they're more useful if you run them in YOLO mode rather than having to approve everything they want to do.

Coding agents are still a relatively new category to most people. Claude Code dates back to February last year, and it took a while for the general engineering public to understand why that format - coding LLMs that can execute and iterate on the code they are writing - was such a big deal.

As a result the demand for good sandboxing options is skyrocketing.

It also takes a while for new solutions to spin up - if someone realized sandboxes were a good commercial idea back in September last year the products they built may only just be ready for people to start trying out today.

Re: Nobody likes lag: How to make low-latency dev sandboxes

#25

Post by ambicapter »

simonw wrote: post #24
iterateoften wrote: post #17

Why is there all the sudden an explosion of sandbox related posts and tools? Llms and agents always needed sandboxes… was it just the collective conscious decided all at once that it mattered and the area to focus building tools?

I think sandboxes are having their moment because it's become undeniable that coding agents are useful, and that they're more useful if you run them in YOLO mode rather than having to approve everything they want to do. Coding agents are still a relatively new category to most people. Claude Code dates back to February last year, and it took a while for the general engineering public to understand why that format - c…

Why/how are they more useful in YOLO mode than in careful mode?

Re: Nobody likes lag: How to make low-latency dev sandboxes

#26

Post by phainopepla2 »

ambicapter wrote: post #25
simonw wrote: post #24

Earlier quoted context omitted.

I think sandboxes are having their moment because it's become undeniable that coding agents are useful, and that they're more useful if you run them in YOLO mode rather than having to approve everything they want to do. Coding agents are still a relatively new category to most people. Claude Code dates back to February last year, and it took a while for the general engineering public to understand why that format - c…

Why/how are they more useful in YOLO mode than in careful mode?

It's just a lot easier to let them run loose and finish a task before reviewing it, rather than have to babysit and approve every command they want to run. It frees you up to do other things in that time. For some people, that's running more agents in a different terminal, for others that doing something else entirely.

Re: Nobody likes lag: How to make low-latency dev sandboxes

#27

Post by nicolaslecomte »

Thanks for sharing. Makes a lot of sense that removing that routing layer would improve e2e latency.

We had a similar bottleneck building out our sandbox routing layer, where we were doing a lookup to a centralized db to route the query. We found that even with a fast KV store, that lookup still added too much overhead. We moved to encoding the routing logic (like region, cluster ID, etc) directly into the subdomain/hostname. This allowed to drop the db read entirely on the hot path and rely on Anycast + latency-based DNS to route the user to the exact right regional gateway instantly. Also, if you ever find yourselves outgrowing standard HTTP proxies for those long-lived agent sessions, I highly recommend looking at Pingora. It gave us way more control over connection lifecycles than NGINX.

For the compute aspect doing sandbox pooling is cool but might kill your unit economics, especially if at some point each tenant has different images. Have you looked into memory snapshots (that way you only have storage costs not full VMs)?

Re: Nobody likes lag: How to make low-latency dev sandboxes

#28

Post by imiric »

So they used edge servers? How is this novel or insightful?

This article reads like a thinly veiled ad. Certainly not the best way to start a technical blog. If you didn't have the technical insight to know that physics is a factor in latency, why should I trust you with the problems your product actually solves?

Re: Nobody likes lag: How to make low-latency dev sandboxes

#30

Post by esperent »

ambicapter wrote: post #25
simonw wrote: post #24

Earlier quoted context omitted.

I think sandboxes are having their moment because it's become undeniable that coding agents are useful, and that they're more useful if you run them in YOLO mode rather than having to approve everything they want to do. Coding agents are still a relatively new category to most people. Claude Code dates back to February last year, and it took a while for the general engineering public to understand why that format - c…

Why/how are they more useful in YOLO mode than in careful mode?

The flow I'm using is plan -> technical plan -> execute using TDD.

My level of involvement decreases from step to step. I'm totally in control of the initial plan. I'm giving strong oversight of the technical plan. But by the time it comes to executing, I'm happy to let it completely take over and I'll review either at the end, or break it down into 2 - 4 phases for long plans and I'll review after each phases.

For this final step, which might be 30 minutes, I'll step out and do something else. I want to be sure nothing bad will happen on my machine if I do that, so sandboxing is important.

Post reply on HN