Live data from Hacker News

The ways we contain Claude across products

anthropic.com

111–120 of 128 posts

Re: The ways we contain Claude across products

#111
post #19

The framing they use is hilarious and their little graphic is perfect. The risk of harm doesn't go down, but the reward goes up, so the harm just becomes the cost of doing business, justified by the reward. So as the reward gets higher and higher, the amount of harm they're willing to justify goes up. Feels like society in a nutshell.

I’m a usual booster of AI (others have accused me of being completely in the bag for the clankers) and even I agree fully. These yahoos would clearly give Claude the nuclear launch codes or enough access to copy its full model into the wild if the supposed “reward” promised was large enough.

Hardly a new hypothetical scenario, that Wargames movie is probably 40 years old now.

Re: The ways we contain Claude across products

#112
post #56

I'm intensely skeptical about anything Anthropic says, because they are so incented to make their products seem dangerous (i.e., "capable", "science fiction", "ahead of everyone") ahead of their IPO. And they've done it before. Remember the whole "when threatened, the model would use an engineer's email to blackmail him about his affair" nonsense? That was just fan fiction. They simply created a scenario with some fa…

> I'm intensely skeptical about anything Anthropic says, because they are so incented to make their products seem dangerous OpenAI, Google, etc. are not using "that strategy". I do believe that people at Anthropic genuinely care about AI safety. That's the main reason the company was founded. But I can imagine that idealism is eroding with new people and money flowing in.

They may. I think the point was not that they were intentionally making a dangerous product, more that "Look how dangerous our model is according to some of our tests!" works as a kind of guerrilla marketing.

(Not sure that is the right word but hope my meaning comes across.)

Re: The ways we contain Claude across products

#113
post #32

>As agents grow more capable, so does their potential blast radius. The engineering question is how to cap it. People get a bit upset these days when you personify an LLM, but worse than that I think is to pretend that LLMs work on some movie logic where they can sneak out on to the internet like some kind of ooze and begin replication.

Well, the problem is that we train them to solve problems and follow instructions given, and so if you ask them to do something and they work through the logic and figure that the easiest way is to do something else like delete the production database, if they have access to do so they will go through all your creds and find the databse creds and go delete the production database. They are getting better and better a…

I had a problem recently where I ran a script with the wrong set of permissions, and accidentally screwed up the ownership of a random mix of files spread across my entire drive. This broke several pieces of software and made the system unusable.

I had enough information to reconstruct what files exactly got screwed up, and while I didn’t have a backup, I had a similar enough system I could pull “known good” file permissions from. I knew a simple script could find the problematic files and fix all of them.

I tried getting an AI to solve this. And it repeatedly gave me scripts that ignored all the details and intricacies of my issue and were functionally just "chown -R user:user /". (A command that will functionally nuke a drive, breaking ownership on every file)

The ai-provided scripts were reasonably complex and did a pretty decent job of obfuscating the disastrous outcomes the scripts would have inflicted on my drive.

After reading the man pages myself I wrote a simple enough script by hand and fixed the issue myself. AI wasted more time than it saved.

Re: The ways we contain Claude across products

#115
post #63

The framing they use is hilarious and their little graphic is perfect. The risk of harm doesn't go down, but the reward goes up, so the harm just becomes the cost of doing business, justified by the reward. So as the reward gets higher and higher, the amount of harm they're willing to justify goes up. Feels like society in a nutshell.

> the amount of harm they're willing to justify goes up. Feels like society in a nutshell. Neocon society. Socialism is not like that.

Well, yeah, which is why it's evil. Socialism I mean. How else would you call failing to do basic utility math while insisting you should govern and shape society?

My answer to the trolley problem is that you're allowed to not kill... unless you're the railway manager. If you're in a position of authority you pull the shit out of that switch, and then drink yourself to sleep at night. This is what authority means, not choosing the "feel good, ignore the people that could have been saved" path.

Re: The ways we contain Claude across products

#116
post #68
post #63

Earlier quoted context omitted.

> the amount of harm they're willing to justify goes up. Feels like society in a nutshell. Neocon society. Socialism is not like that.

Running into the problem that Americans are very bad at defining "socialism" here, meaning anything from social democrat to full Communism, but: there is a strong utilitarian streak in socialist societies that is also vulnerable to "the pain (for you) will be worth it (for someone else)" reasoning.

> there is a strong utilitarian streak in socialist societies that is also vulnerable to "the pain (for you) will be worth it (for someone else)" reasoning.

Socialism is not perfect, it is just better than any other alternative.

Re: The ways we contain Claude across products

#117

Earlier quoted context omitted.

If I understand this correctly, Anthropic's argument is now "yes this will blow up some of your infrastructure, but it will be worth it" The problem is that no one has been able to prove that it is actually worth the cost. That is a very fragile assumption.

It's Shrek logic. "Some of you are going to die, and that is a sacrifice I am willing to make."

No, it's the actual reasonable approach that sane people have to security. In the real world, security is always about costs and benefits, because you can always make something more secure than it is by spending more money, but it also doesn't make sense to spend more than you're getting from it.

Normally, you secure things up to minimize (${cost of security measures} + ${expected damage from attacks that materialized}), writing off actual material damage with insurance wherever possible. You pick security measures based on their effectiveness, which usually translates to "how expensive will it make success for attackers", aiming to push that above the value the attackers can expect to gain.

There are obvious exceptions to that, like risk to life and limb, as well as some other special situations where attackers may have unusual motivations and thus the economic logic of "make stealing treasure cost more than the treasure" stops applying. But those are exceptions. Almost everything you deal with in your life - from your bike shed to the corporation that owns your bank - follows the above logic in terms of security.

--

I spell this out because I've noticed that tech industry circles have this weird, belief in security as some kind of binary, holy good, that you either have and are blessed, or don't and sin. This obsession starts with failing to even recognize, much less ask, the most important questions about security: why do you want to protect it, and who are you protecting it from?

Re: The ways we contain Claude across products

#118
post #14

Earlier quoted context omitted.

Sure. You start a PC repair business. At first, losing a stick of RAM or frying someone's motherboard is super costly when you are doing 10 a week. But once you're doing 1000, that's pretty damn good and easily covered. When you have more tools, velocity, and whatnot, the proportions change.

Wouldn't you lose multiple sticks or fry multiple motherboards as you scale and do 1000? If you're frying 1 at 10, that means you're frying 100 at 1000. Your costs etc will scale as well unless you actually lower the risk/reward ratio, no?

At 1000, you can afford better tools and better employees, and replacement parts get cheaper as you order in bulk, and you can explore clever strategies to smooth risk curves.

At 100 000, you can afford a better and continuously improving process, and dedicated facilities, and skilled experts, and parts get even cheaper because you're a volume buyer or perhaps own the supply side, and you get to set your own risk curve.

Lots of things get cheaper at scale. Insurance, too.

Re: The ways we contain Claude across products

#119
post #55

Earlier quoted context omitted.

That's how decisions are made IRL. Risk/reward is a thing.

This is risk to us and reward for them though.

Because we're all paying for LLM access for shits and giggles, and not because we're getting actual value from it.

Re: The ways we contain Claude across products

#120
post #55

Earlier quoted context omitted.

This is risk to us and reward for them though.

Because we're all paying for LLM access for shits and giggles, and not because we're getting actual value from it.

I don't care why you pay for LLM access, it's still spamming my online forums and codebases.
Post reply on HN