51–60 of 61 posts
Earlier quoted context omitted.
Evidence?
I worked on this for a company that got bought by one of the labs (for more than just agent sandboxes, mind you).
If you did not solve it why were you bought?
Earlier quoted context omitted.
The risk isn't from the AI labs. It's from malicious attackers who sneak instructions to coding agents that cause them to steal your data, including your environment variable secrets - or cause them to perform destructive or otherwise harmful actions using the permissions that you've granted to them.
Simon, I know you're the AI bigwig but I'm not sure that's correct. I know that's the "story" (but maybe just where the AI labs would prefer we look?). How realistic is it really that MCP/tools/web search is being corrupted by people to steal prompts/convos like this? I really think this is such low prop. And if it does happen, the flaw is the AI labs for letting something like this occur. Respect for your writing, b…
Earlier quoted context omitted.
Right, but the multiply step is not AI specific. Let's focus here: AI providers farming out their convos to 3rd-parties? Unlikely, but if it happens, it's totally their bad. I really don't think this is a thing.
Right, but this is still a hygiene issue, if you are skipping washing your hands after using the bathroom because its unlikely that the bathroom attendants didn't clean it up you are going to have a bad time.
I just think this whole thing is overblown.
If there's a risk in any situation it's similar, probably less, than running any library you installed of a registry for your code. And I think that's a good comparison: supply chain is more important than AI chain.
You can consider AI-agents to be like the fancy bathrooms in a high end hotel, whereas all that code you're putting on your computer? That's the grimy public lavatory lol.
[flagged]
> a secrets store that the model can "use" but never "read". How would that work? If the AI can use it, it can read it. E.g: secret-store "foo" > file cat file You'd have to be very specific about how the secret can be used in order for the AI to not be able to figure out what it is. You could provide a http proxy in the sandbox that injects a HTTP header to include the secret, when the secret is for accessing a webs…
I'm working on something similar called agent-creds [0]. I'm using Envoy as the transparent (MITM) proxy and macaroons for credentials. The idea is that you can arbitrarily scope down credentials with macaroons, both in terms of scope (only certain endpoints) and time. This really limits the damage that an agent can do, but also means that if your credentials are leaked they are already expired within a few minutes.…
How would you build such an authz scheme? When claude asks permissions to access a new endpoint, if the user allows it, then reissue the macaroons?
I'm working on something similar called agent-creds [0]. I'm using Envoy as the transparent (MITM) proxy and macaroons for credentials. The idea is that you can arbitrarily scope down credentials with macaroons, both in terms of scope (only certain endpoints) and time. This really limits the damage that an agent can do, but also means that if your credentials are leaked they are already expired within a few minutes.…
> With macaroons you can design the authz scheme that you want for any arbitrary API. How would you build such an authz scheme? When claude asks permissions to access a new endpoint, if the user allows it, then reissue the macaroons?
1. You can issue your own tokens which means you can design your own authz in front of the upstream API token.
2. Macaroons can be attenuated locally.
So at the time that you decide you want to proxy an upstream API, you can add restrictions like endpoint path to your scheme.
Then, once you have that authz scheme in place, the developer (or agent) can attenuate permissions within that authz scheme for a particular issued macaroon.
I could grant my dev machine the ability to access e.g. /api/customers and /api/products. If i want to have claude write a script to add some metadata to my products, I might attenuate my token to /api/products only and put that in the env file for the script.
Now claude can do development on the endpoint, the token is useless if leaked, and Claude can't read my customer info.
Stripe actually does offer granular authz and short lived tokens, but the friction of minting them means that people don't scope tokens down as much.
Earlier quoted context omitted.
I worked on this for a company that got bought by one of the labs (for more than just agent sandboxes, mind you).
Wait, let me get this straight: “there’s no solution” to this apparent giant problem but you work for a company that got bought by an AI corp because you had a solution? Make it make sense. If you did not solve it why were you bought?
Earlier quoted context omitted.
Wait, let me get this straight: “there’s no solution” to this apparent giant problem but you work for a company that got bought by an AI corp because you had a solution? Make it make sense. If you did not solve it why were you bought?
I worked for a company that got bought because they were working on a number of problems of interest to the acquirer. As many of these were hard problems, our efforts on them and progress was more than enough.
Earlier quoted context omitted.
> With macaroons you can design the authz scheme that you want for any arbitrary API. How would you build such an authz scheme? When claude asks permissions to access a new endpoint, if the user allows it, then reissue the macaroons?
There are two parts here: 1. You can issue your own tokens which means you can design your own authz in front of the upstream API token. 2. Macaroons can be attenuated locally. So at the time that you decide you want to proxy an upstream API, you can add restrictions like endpoint path to your scheme. Then, once you have that authz scheme in place, the developer (or agent) can attenuate permissions within that authz…
For example, how do you collect all the endpoints that have access to customer info per your example.
Thought about it and couldn't find a way how
Earlier quoted context omitted.
There are two parts here: 1. You can issue your own tokens which means you can design your own authz in front of the upstream API token. 2. Macaroons can be attenuated locally. So at the time that you decide you want to proxy an upstream API, you can add restrictions like endpoint path to your scheme. Then, once you have that authz scheme in place, the developer (or agent) can attenuate permissions within that authz…
I understand that, but how do you come up with the endpoints you want claude to have access to ahead of time? For example, how do you collect all the endpoints that have access to customer info per your example. Thought about it and couldn't find a way how
For example, If I want to write a skill that can pull subscription cancellations from today, research the cancellation reason, and then push a draft email to gmail, then ideally I'd have...
- a 5 minute read-only token for /subscriptions and /customers for stripe
- a 5 minute read-write token to push to gmail drafts
- a 5 minute read-only token to customer events in the last 24h
Claude understands these APIs well (or can research the docs) so it isn't a big lift to rebuild authz, and worst case you can do it by path prefix and method (GET, POST, etc) which works well for a lot of public APIs.
I feel like exposing the API capability is the easy part, and being able to get tight-fitting principle-of-least-privilege tokens is the hard part.