Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The ways we contain Claude across products

anthropic.com

Post reply on HN
Print view View original

101–110 of 128 posts

Re: The ways we contain Claude across products

#101

Post by pixl97 »

vrganj wrote: post #81
jon-wood wrote: post #79

Earlier quoted context omitted.

Everything you do a risk/reward equation, you just don't usually see it drawn out quite so starkly. Getting out of bed in the morning carries a risk that you'll trip and crack your head on the floor. Crossing a road carries a risk of being hit by a bus. Eating food carries a risk of choking on it. The same is true in computer security. The only truly secure computer is one you don't turn on, and even that carries som…

But if you eat food, I don't risk choking. They want us to take the risk for their reward.

But if I drive a car, You do run the risk of getting ran over. We can come up with any number of analogies of varying rightness and wrongness here.

Re: The ways we contain Claude across products

#102

Post by pixl97 »

protocolture wrote: post #27

>As agents grow more capable, so does their potential blast radius. The engineering question is how to cap it. People get a bit upset these days when you personify an LLM, but worse than that I think is to pretend that LLMs work on some movie logic where they can sneak out on to the internet like some kind of ooze and begin replication.

> that LLMs work on some movie logic where they can sneak out on to the internet like some kind of ooze and begin replication.

Why not? If you're not talking about running the model itself, AI agents are perfectly capable of writing an agent worm capable of spreading more agents around via software exploits.

Now, currently LLMs are too hardware intensive to spread the model itself, but given a few years and optimizations we may very well see that too.

What you're saying reminds me of the old days when people said things like "images can't spread viruses", then suddenly people found decoder vulns and made image viruses that did exactly that.

Re: The ways we contain Claude across products

#103

Post by yencabulator »

> The proxy sits inside the VM rather than on our servers because only the VM knows provenance—from the server's perspective, a Cowork request is indistinguishable from any other API client.

That means the attacker can still exfiltrate files if they get root inside the VM.

Why not run the proxy outside the VM, still on the client?

Re: The ways we contain Claude across products

#106

Post by KaiShips »

barrkel wrote: post #67
andai wrote: post #16

Earlier quoted context omitted.

Yeah I was thinking about Simon Wilson's "lethal trifecta"[0] in the context of OpenClaw style "general purpose" AI agents, where people just gave it access to their full hard drive, gmail account, etc. I was thinking you can't make the chance of catastrophic failure zero (we still hear about "Claude deleted my home folder"), but you can definitely limit the blast radius. You can't get the risk to zero. But the oppor…

Containment of the execution environment isn't really the issue. It's API tokens that were designed with coarse permission scoping so agents get more power than they need. The risk isn't that your machine gets hacked. It's that your email gets deleted, or forwarded to someone who uses it to break into your other accounts via password recovery.

[flagged]

Re: The ways we contain Claude across products

#107

Post by daveshistory »

soundworlds wrote: post #44
6gvONxR4sf7o wrote: post #9

The framing they use is hilarious and their little graphic is perfect. The risk of harm doesn't go down, but the reward goes up, so the harm just becomes the cost of doing business, justified by the reward. So as the reward gets higher and higher, the amount of harm they're willing to justify goes up. Feels like society in a nutshell.

If I understand this correctly, Anthropic's argument is now "yes this will blow up some of your infrastructure, but it will be worth it" The problem is that no one has been able to prove that it is actually worth the cost. That is a very fragile assumption.

It's Shrek logic. "Some of you are going to die, and that is a sacrifice I am willing to make."

Re: The ways we contain Claude across products

#108

Post by daveshistory »

pixl97 wrote: post #101
vrganj wrote: post #81

Earlier quoted context omitted.

But if you eat food, I don't risk choking. They want us to take the risk for their reward.

But if I drive a car, You do run the risk of getting ran over. We can come up with any number of analogies of varying rightness and wrongness here.

And then there's a whole truckload of case law about liability that comes into play.

We haven't yet written those laws for "AI."

Re: The ways we contain Claude across products

#109

Post by daveshistory »

enraged_camel wrote: post #66
chrncirurp wrote: post #53

Earlier quoted context omitted.

> I think the point is that at small scale a single accident poses a risk of ruin to your small operations. At big scale, a single big accident poses a risk to ruin your big operations.

No, it does not. Every large company eventually has a big accident. They survive because they have both the resources (e.g. to fight ensuing legal battles, or pay fines, or simply weather a hit to reputation and the resulting downturn in revenue) as well as redundancy, different types of insurance, and so on.

Companies of all sizes should have insurance to cover such scenarios. You need to get tradesman's insurance on your repair work, or you need to ask yourself why the insurance companies won't insure you.

Re: The ways we contain Claude across products

#110

Post by daveshistory »

vrganj wrote: post #55
solenoid0937 wrote: post #33

Earlier quoted context omitted.

That's how decisions are made IRL. Risk/reward is a thing.

This is risk to us and reward for them though.

Many companies would say that's the best kind of risk-reward balance. For them, anyway.
Post reply on HN