Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's Safety Superpower

stratechery.com

Post reply on HN
Print view View original

21–30 of 206 posts

Re: Anthropic's Safety Superpower

#21

Post by keybored »

> Here’s the thing about these safety justifications: I think they work because, to Anthropic, they aren’t justifications. The company really believes that they are the only ones who believe in super intelligence, and thus are the only ones who are sufficiently concerned about the dangers. That excuses decision after decision, policy after policy, and confrontation after confrontation that, to people on the outside, look like a bizarre combination of cynicism and naiveté.

I really dislike this belief (that has at least been expressed here) by some that X is okay because they-really-believe-it. This has a real Road to Hell stank on it.

It is incredibly convenient when your predictions or supposed beliefs go south. Well, we really believed that we were doing it for the betterment of human kind. And we really believed that X was an existential threat that was inevitable in which case we had to step up and do it because we we the only good guy ideologues. So sorry but not sorry.

I also don’t care if commenters know rank-and-file on the inside that “really believe it” as well. Not for one second.

Re: Anthropic's Safety Superpower

#22

Post by re-thc »

s_dev wrote: post #18
ramon156 wrote: post #11

Earlier quoted context omitted.

No. If we cannot even have an EU CloudFlare, then we definitely do not have the infra for this kind of computing. The EU options are not even close to what CF can do

>EU CloudFlare What limitations does bunny.net have?

> What limitations does bunny.net have?

A huge free tier (technically, none)

Re: Anthropic's Safety Superpower

#23

Post by swalsh »

"they by extension think that only they should have final say over AI generally. When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."

That might be one of the most important points in the post. Very troubling.

Re: Anthropic's Safety Superpower

#24

Post by mcintyre1994 »

cube2222 wrote: post #4

Relatedly, I think it's worth noting that Anthropic models have consistently been top-scoring in BullshitBench[0], in a league of their own, really. Not affiliated with the bench in any way, but I think it surfaces important differences between the behavior of the models from different labs. TLDR: The benchmark is measuring pushback in response to nonsensical requests and questions, as opposed to going with it and ha…

TBH this is the main thing that made me start trusting Claude enough to actually find it useful, and I'm surprised other models haven't caught up. I assumed they had and I just wasn't aware because I'm not using them in the same way.

Re: Anthropic's Safety Superpower

#25

Post by re-thc »

Zealotux wrote: post #8
smackeyacky wrote: post #5

Perhaps they should consider leaving the US. Pretty clearly the descent into a corrupt autocracy is having real consequences.

Does any other place have the infrastructure Anthropic requires to train their models and run inference?

> Does any other place have the infrastructure

That's not the problem.

The US government can export ban GPUs like they do now to more countries if needed. Even if the infrastructure exists, the GPUs won't.

Re: Anthropic's Safety Superpower

#26

Post by chasil »

khalic wrote: post #10
chasil wrote: post #3

(reposted) As I understand it, ITAR regulations for export controls have just been applied to any form of Mythos. These are overseen by U.S. Departments of State and Commerce, and forbid foreign nationals from access to any form of Mythos, either within or outside the U.S. Only U.S. citizens and immigrants that are holders of a "green card" may now access Mythos. It appears that Anthropic does not have internal contr…

This is how the US gov does business now, capricious and vengeful. Textbook retaliation for not letting them use an abliterated version of Claude in weapons systems. This effectively renders any US closed model useless for any foreign company. Could happen to OpenAI, Google, etc. Too much of a risk to implement something that can be yanked out because the company didn’t behave the way they want. Looks like it’s time…

Consider this quote from the main article...

"When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."

This is fearful stuff on all sides, and none of the people involved might realistically be able to navigate the danger.

Re: Anthropic's Safety Superpower

#27

Post by chasil »

WithinReason wrote: post #16
chasil wrote: post #3

(reposted) As I understand it, ITAR regulations for export controls have just been applied to any form of Mythos. These are overseen by U.S. Departments of State and Commerce, and forbid foreign nationals from access to any form of Mythos, either within or outside the U.S. Only U.S. citizens and immigrants that are holders of a "green card" may now access Mythos. It appears that Anthropic does not have internal contr…

Could Anthropic relocate to a different country?

Individuals can leave, but the company cannot transfer restricted intellectual property.

Europe has extradition treaties, so the U.S. can force anyone in Europe back to the U.S. for criminal indictment who demonstrates inappropriate possession of this technology.

Re: Anthropic's Safety Superpower

#28

Post by vidarh »

eloisant wrote: post #17
chasil wrote: post #3

(reposted) As I understand it, ITAR regulations for export controls have just been applied to any form of Mythos. These are overseen by U.S. Departments of State and Commerce, and forbid foreign nationals from access to any form of Mythos, either within or outside the U.S. Only U.S. citizens and immigrants that are holders of a "green card" may now access Mythos. It appears that Anthropic does not have internal contr…

I never really understood this "US person" restriction. There are 350M people in US, mostly citizens and green cards holders, surely some of them could be working for a foreign power.

They don't even need to know they are. You can assume that if the model becomes available again, a lot of people will find themselves working for companies distilling these models that just happens to ultimately do work for foreign entities, whether or not the people accessing the models knows or not.

Re: Anthropic's Safety Superpower

#29

Post by dofm »

thefounder wrote: post #19
olmo23 wrote: post #12

Earlier quoted context omitted.

> no nationality controls in place Not for now, but how long before we have KYC regulations concerning LLMs?

That’s really what Dario wants. Let’s hope he doesn’t get it

Regulatory capture is the OpenAI and Anthropic end goal, for certain.

But I also think they exist in a sort of un-designed corporate narcissism, which is a common trait in bubble economies — I am not judging them particularly severely.

Netscape under Clark and Andreessen and Sun under McNealy both fell into corporate narcissism: the belief that only they really mattered, that they were chosen, and that the world needed to rearrange itself to just let them shine. They arguably let themselves get played by Oracle (a corporate psychopath) and others as a result.

OpenAI's position is profoundly corporate-narcissistic: all we need is all the money in the economy and not to have to do anything upsetting like think about turning a profit for the next four years. Like rich kids. It would be nice if you believed we were so important that we should get an enormous stipend for just being us.

Anthropic's position is: we think we're so unique and ominous that government needs to make us both essential and terrifying. We have to exist otherwise worse people will.

Both narcissistic positions.

Re: Anthropic's Safety Superpower

#30

Post by LoganDark »

> The entire Anthropic origin story is rooted in the founders’ belief that OpenAI wasn’t taking safety seriously enough; the company believes that only they can control AI, and that because they uniquely care about safety, they are justified in trying to control everyone else, up to and including the U.S. government.

Anthropic believes they have the responsibility to guard their tools from mis-use. That is all. They are not trying to "control" anything or anyone. They do however decide what they think is mis-use.

Post reply on HN