[dead]
The agent harness belongs outside the sandbox
81–90 of 140 posts
Re: The agent harness belongs outside the sandbox
#82There are other models. Eschew the sandbox. Give the agent a computer, with all the trimmings, but keep that computer segregated from sensitive resources. Tokens are a solved problem: tokenize them[1] or do something equivalent with a proxy. The same thing goes for secrets. A lot of this post presents false dichotomies. It assumes the existence of a sandbox that is by definition ephemeral or "cattle-like". Why? There…
Wow, thanks for writing this up.
I'm building an agent sandboxing system for a client atm, and was about to start working on a system of ephemeral, short lived, derived secrets for the agent to use.
Lots of great thoughts to steal in this piece. Thanks again.
Re: The agent harness belongs outside the sandbox
#83Earlier quoted context omitted.
Author here. In my opinion, the main driver here is how fast models have evolved in the past 12 months. It makes the architecture of everything around them obsolete, very fast. We went from using models as a building block, wrapping them in heavy workflow code, to now models being smart enough to drive their own workflows and planning.
Really enjoyed your post, by the way. The idea of putting skills and memories in a database while keeping the file shaped interface for the agent is clean. One read/write surface, two backends, invisible to the modle that's a nice piece of design, and the candor in the "what's still hard" section made me trust the rest of the post. My comment above was meant as a joke, not about your architecture. If this pattern bec…
Thank you, appreciate it!
Regarding scoping: In our case, the agent loop runs in the same way as our API server does (as in, it’s a multi tenant service running in a container somewhere). And we solve scoping in the same way.
To put it in other words, whether it’s the API receiving “GET /memories/id” or the LLM requesting “Read(/memories/id)” we do pretty much the same thing (check authN/authZ, scope the db request, etc).
Basically the LLM is just another API client using a slightly different format for inputs and outputs, but sharing the same permission layer.
Re: The agent harness belongs outside the sandbox
#84I think it omits the real reason I want to run the harness in the sandbox: I barely trust the harness more than the LLM, at least at this point in time. They are so rapidly evolving along with the underlying models, that I don't think they are a reasonable component to rely on to provide safety constraints. Put more precisely: if your harness has an ability to do something the LLM can't, and it has a set of condition…
> Effectively you have an arm of the lethal trifecta and pretending otherwise is more dangerous than helpful.
"Lethal trifecta" is basically describing phishing but in a way more palatable to people who would rather die before allowing themselves to anthropomorphize LLMs even a little bit. It's not a problem you can fix with better coding, like some SQL injection. You can only manage risk around it (for which sandboxing is one of many solutions that can help).
So on one hand, I agree with you - you need to be mindful of what you're actually dealing with. On the other hand, you always have this, and need this, for the agent to be able to do anything useful.
Re: The agent harness belongs outside the sandbox
#85I think it omits the real reason I want to run the harness in the sandbox: I barely trust the harness more than the LLM, at least at this point in time. They are so rapidly evolving along with the underlying models, that I don't think they are a reasonable component to rely on to provide safety constraints. Put more precisely: if your harness has an ability to do something the LLM can't, and it has a set of condition…
> if your harness has an ability to do something the LLM can't What does this even mean. The only capability of an LLM is generate text.
The LLM can only generate text. The harness can do more than just generate text. By joining the two you're allowing the LLM (through text) to carry out whatever actions the harness can take.
My brain can only generate electrical signals. My hand responds to electrical signals and can interact with the real world. The two together can do more than just what my brain alone can do.
If you don't trust a particular brain, don't put a gun in the hand which is connected to it. If you don't trust a LLM, don't connect it to a harness which has access to your production database and only recent backups (https://www.theregister.com/2026/04/27/cursoropus_agent_snuf...).
Re: The agent harness belongs outside the sandbox
#86Earlier quoted context omitted.
Yes, the concept itself is not new. Around 2022, people would usually have called it the orchestration layer. But I think the term started being used closer to its current meaning around this point: https://www.softwareimprovementgroup.com/blog/what-is-harnes... In a way, the sequence was something like: prompt engineering(23~4) -> context engineering(25) ->harness engineering(26) At first, it was mostly understood a…
My 2c: The word harness brings the truth of LLMs back down to Earth. it really felt like between 2018 and 2022ish like LLMs had this magical aura, like the orchestration layer was intelligent, maybe even recursive, beyond what simple functions could do. It was assumed that this was a solved problem. The word "orchestration" denoted it, the words we used were full of optimism. When you lift the veil, it really is just…
There were no LLMs between 2018 and 2022, at least not in the sense resembling today. The whole LLM frenzy started in late 2022.
Re: The agent harness belongs outside the sandbox
#87[flagged]
Re: The agent harness belongs outside the sandbox
#88Agreed, this is exactly what we do.
There's no harm in a string, only in the execution.
I create Tools as Actors, which you preconfigured for the LLM context (in-house agent loop). The tools being preconfigured means you setup their environment before they can be executed. If it calls a bash tool for instance, the Tool Actor gets called and then it runs that command against an attached remote VM.
Or filesystem operations, are just read/writes inside a .zip file, which is overlayed onto the target project at build time.
This article is spot on, and I probably say that because it's self reinforcing.
Re: The agent harness belongs outside the sandbox
#89I think it omits the real reason I want to run the harness in the sandbox: I barely trust the harness more than the LLM, at least at this point in time. They are so rapidly evolving along with the underlying models, that I don't think they are a reasonable component to rely on to provide safety constraints. Put more precisely: if your harness has an ability to do something the LLM can't, and it has a set of condition…
> if your harness has an ability to do something the LLM can't What does this even mean. The only capability of an LLM is generate text.
We’ve trained models on JSON schemas for “tool calls”, and then built software to interpret and run those calls for the LLMs
Re: The agent harness belongs outside the sandbox
#90[dead]