Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

171–180 of 716 posts

Re: Anthropic drops flagship safety pledge

#171
post #118

Earlier quoted context omitted.

Neither of them had shareholders to please.

I don't believe anthropic has shareholders either. It is not a public company

Still has private investors it can't ignore, until it can buy them out, but it can't do that until it starts turning over a profit. Even then it may not be able to get rid of them if they own enough of a share.

Re: Anthropic drops flagship safety pledge

#173
post #151

> “We felt that it wouldn't actually help anyone for us to stop training AI models,” How magnanimous! They are only thinking of others, you see. They are rejecting their safety pledge for you . > “We didn't really feel, with the rapid advance of AI, that it made sense for us to make unilateral commitments … if competitors are blazing ahead.” Oops, said the quiet part out loud that it’s all about money. “I mean, if al…

[flagged]

The most important part of AI safety is AI alignment: making sure AI does what we want. It's very hard because even if AI isn't trying to deceive you it can have bad outcomes by executing your request to the letter. The classical example is tasking an AI to make paperclips, training the AI with a reward for making more paperclips. Then the AI makes the most paperclips possible by strip mining the Earth and killing anything in its way.

Sometimes you see this AI alignment problem in action. I once asked an older model to fix the tests and it eventually gave up and just deleted them

Re: Anthropic drops flagship safety pledge

#174

I don't understand how safety is taken seriously at all. To be clear, I'm not referring to skepticism that these companies can possibly resist the temptation to make unsafe models forever. No, I'm talking about something far more basic: the fact that for all the talk around safety, there is very little discussion about what exactly "safety" means or what constitutes "ethical" or "aligned" behavior. I've read reams of…

Related:

[0]. https://civai.org/p/ai-values

Re: Anthropic drops flagship safety pledge

#175
post #151

> “We felt that it wouldn't actually help anyone for us to stop training AI models,” How magnanimous! They are only thinking of others, you see. They are rejecting their safety pledge for you . > “We didn't really feel, with the rapid advance of AI, that it made sense for us to make unilateral commitments … if competitors are blazing ahead.” Oops, said the quiet part out loud that it’s all about money. “I mean, if al…

> Oops, said the quiet part out loud that it’s all about money. “I mean, if all of our competitors are kicking puppies in the face, it doesn’t make sense for us to not do it too. Maybe we’ll also kick kittens while we’re at it”.

I mean, yes, that is actually how world works. That is why we need safety, environmental and other anti-fraud regulations. Because without them, competition makes it so that every successful company will fraud, hurt and harm. Those who wont will be taken over by those who do.

Re: Anthropic drops flagship safety pledge

#176
post #162
post #73

How is this article not going to even mention the recent threats to Anthropic from the Government?!

I mean seriously, is this not the very definition of fascism? "n general, fascist governments exercised control over private property but they did not nationalize it. Scholars also noted that big business developed an increasingly close partnership with the Italian Fascist and German Nazi governments after they took power. Business leaders supported the government's political and military goals. In exchange, the gove…

All governments do this

Re: Anthropic drops flagship safety pledge

#177

Earlier quoted context omitted.

[flagged]

Well we teach kids not to yell “Fire!” In a crowded theatre or “N***!“ at their neighbor. We also teach our industrial machines to distinguish between fingers and bolts, our cars to not say “make a left turn now” when on a bridge, etc

The critical point is who the "we" is.

Is "we" the parents teaching their children their own unique values, or is the "we" a government or corporation forcing one set of values on all children.

Why not encourage the users of AI to use a Safety.md (populated with some reasonable but optional defaults)?

Re: Anthropic drops flagship safety pledge

#178
post #128

Earlier quoted context omitted.

I don’t get it. Even the Soviet Union used money. Simply paying for stuff isn’t necessarily capitalism? Or are you suggesting Anthropic should be state-owned?

Using money as a medium to facilitate exchange of goods and services is not capitalism. Abandoning one of your core principles in the pursuit of money, or more charitably because not doing so means your competitors will make more money and overtake you in the marketplace is an outgrowth of capitalism In the Soviet Union the reasons might have been "to beat the Capitalists", "for the pride of our country" or "Stalin a…

>Though a variant of the last one may well have happened here, and the justification we read is just the one less damaging to everyone involved

Hegseth was planning on getting the model via the Defense Production Act or killing Anthropic via supply chain risk classification preventing any other company working with the Pentagon from working with Anthropic. So while it wasn't Siberia, it was about as close as the US can get without declaring Claude a terrorist. Which I'm sure is on the table regardless

Re: Anthropic drops flagship safety pledge

#179
post #175
post #151

> “We felt that it wouldn't actually help anyone for us to stop training AI models,” How magnanimous! They are only thinking of others, you see. They are rejecting their safety pledge for you . > “We didn't really feel, with the rapid advance of AI, that it made sense for us to make unilateral commitments … if competitors are blazing ahead.” Oops, said the quiet part out loud that it’s all about money. “I mean, if al…

> Oops, said the quiet part out loud that it’s all about money. “I mean, if all of our competitors are kicking puppies in the face, it doesn’t make sense for us to not do it too. Maybe we’ll also kick kittens while we’re at it”. I mean, yes, that is actually how world works. That is why we need safety, environmental and other anti-fraud regulations. Because without them, competition makes it so that every successful…

Yes, this. It's unfortunate that anthropic dropped this and it's also exactly how the system is supposed to work. Companies don't regulate themselves, the government regulates the companies.

Now, you may notice that the government is also choosing not to regulate these companies...which is another matter altogether.

Re: Anthropic drops flagship safety pledge

#180

I used to work at Anthropic. I fully believe that the folks mentioned in the article, like Jared Kaplan, are well-intentioned and concerned about the relationship between safety research and frontier capabilities – not purely profit. That said, I'm not thrilled about this. I joined Anthropic with the impression that the responsible scaling policy was a binding pre-commitment for exactly this scenario: they wouldn't s…

If you're not willing to give up your RSUs you shouldn't be surprised that the executives aren't either.

The moral failing is all of ours to share.

Post reply on HN