Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

Post reply on HN
Print view View original

181–190 of 716 posts

Re: Anthropic drops flagship safety pledge

#181

Post by dminik »

rudhdb773b wrote: post #177
gehwartzen wrote: post #163

Earlier quoted context omitted.

Well we teach kids not to yell “Fire!” In a crowded theatre or “N***!“ at their neighbor. We also teach our industrial machines to distinguish between fingers and bolts, our cars to not say “make a left turn now” when on a bridge, etc

The critical point is who the "we" is. Is "we" the parents teaching their children their own unique values, or is the "we" a government or corporation forcing one set of values on all children. Why not encourage the users of AI to use a Safety.md (populated with some reasonable but optional defaults)?

There's nothing a meaningless document can do when the AI is not aligned in the first place.

Re: Anthropic drops flagship safety pledge

#182

Post by isodev »

latexr wrote: post #151

> “We felt that it wouldn't actually help anyone for us to stop training AI models,” How magnanimous! They are only thinking of others, you see. They are rejecting their safety pledge for you . > “We didn't really feel, with the rapid advance of AI, that it made sense for us to make unilateral commitments … if competitors are blazing ahead.” Oops, said the quiet part out loud that it’s all about money. “I mean, if al…

Indeed, Anthropic can’t afford to be the ones that impose any kind of sense in the market - that’s supposed to be the job of the government by creating policy, regulations and installing watchdogs to monitor things.

But lucky for the AI companies, most of them are based in place that only has a government on paper and everyone forgot where that paper is.

Re: Anthropic drops flagship safety pledge

#184

Post by baq »

lebovic wrote: post #156

I used to work at Anthropic. I fully believe that the folks mentioned in the article, like Jared Kaplan, are well-intentioned and concerned about the relationship between safety research and frontier capabilities – not purely profit. That said, I'm not thrilled about this. I joined Anthropic with the impression that the responsible scaling policy was a binding pre-commitment for exactly this scenario: they wouldn't s…

> I hope they're willing to risk losing their seat at the table to be guided by values.

that's about as naive as it can be.

if they have any values left at all (which I hope they have) them not being at the table with labs which don't have any left is much worse than them being there and having a chance to influence at least with the leftovers.

that said, of course money > all else.

Re: Anthropic drops flagship safety pledge

#185

Post by hvsr4z »

lebovic wrote: post #156

I used to work at Anthropic. I fully believe that the folks mentioned in the article, like Jared Kaplan, are well-intentioned and concerned about the relationship between safety research and frontier capabilities – not purely profit. That said, I'm not thrilled about this. I joined Anthropic with the impression that the responsible scaling policy was a binding pre-commitment for exactly this scenario: they wouldn't s…

The EU should invite them over.

The kind of principles you talk about can only be upheld one level up the food chain. By govts.

Which is why legislatures, the supreme court, central banks, power grid regulators deciding the operating voltage and frequency auto emerge in history. Cause corporations structurally cant do what they do without voilating their prime directive of profit maximization.

Re: Anthropic drops flagship safety pledge

#186

Post by nickserv »

isodev wrote: post #182
latexr wrote: post #151

> “We felt that it wouldn't actually help anyone for us to stop training AI models,” How magnanimous! They are only thinking of others, you see. They are rejecting their safety pledge for you . > “We didn't really feel, with the rapid advance of AI, that it made sense for us to make unilateral commitments … if competitors are blazing ahead.” Oops, said the quiet part out loud that it’s all about money. “I mean, if al…

Indeed, Anthropic can’t afford to be the ones that impose any kind of sense in the market - that’s supposed to be the job of the government by creating policy, regulations and installing watchdogs to monitor things. But lucky for the AI companies, most of them are based in place that only has a government on paper and everyone forgot where that paper is.

The government is why they are dropping their pledge.

https://apnews.com/article/anthropic-hegseth-ai-pentagon-mil...

Re: Anthropic drops flagship safety pledge

#187

Post by amelius »

dmix wrote: post #49
heftykoo wrote: post #20

Ah, the classic AI startup lifecycle: We must build a moat to save humanity from AI. Please regulate our open-source competitors for safety. Actually, safety doesn't scale well for our Q3 revenue targets.

Once they are a dominant market leader they will go back to asking the government to regulate based on policy suggestions from non-profits they also fund.

As if their shareholders would agree.

Re: Anthropic drops flagship safety pledge

#188

Post by Valakas_ »

hsbauauvhabzb wrote: post #12
chris_money202 wrote: post #11

First they rushed a model to market without safety checks, and I said nothing. It wasn't my field. Then they ignored the researchers warning about what it could do, and I said nothing. It sounded like science fiction. Then they gave it control of things that matter, power grids, hospitals, weapons, and I said nothing. It seemed to be working fine. Then something went wrong, and no one knew how to stop it, no one had…

Plenty of people have said plenty. The problem isn’t the warnings, it’s that people are too stupid and greedy to think about the long term impacts.

And what makes them being "stupid" and "greedy"? One's intelligence is determined by genes, and greediness is a trait that natural selection has favored for millennia. This is just natural selection taking its course, and it might lead to our end.

If you want to blame something, blame math. Math has determined the physical constants and equations that determine the chemistry and ultimately biology laws that has resulted in humans being the way they are.

Re: Anthropic drops flagship safety pledge

#189

Post by SlinkyOnStairs »

davidguetta wrote: post #160
latexr wrote: post #151

> “We felt that it wouldn't actually help anyone for us to stop training AI models,” How magnanimous! They are only thinking of others, you see. They are rejecting their safety pledge for you . > “We didn't really feel, with the rapid advance of AI, that it made sense for us to make unilateral commitments … if competitors are blazing ahead.” Oops, said the quiet part out loud that it’s all about money. “I mean, if al…

[flagged]

> I VERY LARGELY prefer an AI like grok that doesn't pretend and let the onus of interpretation to the user rather than a bunch of anonymous "researchers" that may be equally biased, at the extreme, may tell you that America's founding father were black women

Setting aside for a moment that Grok is manipulated and biased to a hilarious extent. ("Elon is world champion at everything, including drinking piss")

There is no such thing as "unbiased". There will always be bias in these systems, whether picked up from the training data, or the choices made by the AI's developers/researchers, even if the latter doesn't "intend" to add any bias.

Ignoring this problem doesn't magically create a bias-free AI that "speaks the truth about the founding fathers". The bias in the training data, the implicit unconcious bias in the design decisions, that didn't come out of thin air. It's just somebody else's bias.

All the existing texts on the founding fathers are filled with 250 years of bias, propaganda, and agenda pushing from all sorts of authors.

There is no way to have no bias, no propaganda, no "agenda pushing" in the AI. The only thing that can be done is to acknowledge this problem, and try to steer the system to a neutral position. That will be "agenda pushing" of one's own, but that's the reality of all history and all historians since Herodotus. You just have to be honest about it.

And you will observe that current AI companies are excessively lazy about this. They do not put in the work, but instead slap on a prompt begging the system to "pls be diverse" and try to call it a day. This does not work.

> Of course saying to someone to go kill himslef is a prety sure 'no-no' but so many things are up to interpretation.

Bear in mind that the context of Anthropic's pivot here are the Pentagon's dollars.

This isn't just about "anti-woke AI", it's about killbots.

Sure, Hegseth wants his robots to not do thoughtcrime about, say, trans people or the role of women in the military.

But above all he wants to do a lot of murder.

Antrophic dropping their position of "We shouldn't turn this technology we can barely control into murder machines" because they're running out of money is damnable.

Re: Anthropic drops flagship safety pledge

#190

Post by ben_w »

zer00eyz wrote: post #22
chris_money202 wrote: post #11

First they rushed a model to market without safety checks, and I said nothing. It wasn't my field. Then they ignored the researchers warning about what it could do, and I said nothing. It sounded like science fiction. Then they gave it control of things that matter, power grids, hospitals, weapons, and I said nothing. It seemed to be working fine. Then something went wrong, and no one knew how to stop it, no one had…

> Then something went wrong, and no one knew how to stop it, This is the problem with every AI safety scenario like this. It has a level of detachment from reality that is frankly stark. If linesman stop showing up to work for a week, the power goes out. The US has show that people with "high powered" rifles can shut down the grid. We are far far away from a sort of world where turning AI off is a problem. There isnt…

> We are far far away from a sort of world where turning AI off is a problem. There isnt going to be a HAL or Terminator style situation when the world is still "I, Pencil".

You have to stop the thing before the damage is done.

There are many potential chains of events where the AI has caused enormous damage, and even many where it can destroy us, before the power to its own systems fails.

At this point, with Grok in the Pentagon, just ask what the dumbest military equivalent to vibe-coding is, and imagine the US following that plan.

Like, I dunno, invading Greenland or giving ICE direct control over tactical nukes or something.

And that's just government use. Right now, I'm fairly confident LLMs aren't competent enough to help with anything world-ending unless they get used for war planning by major nuclear powers (oh hey look at the topic of discussion), but it's certainly plausible they'll get good enough at tool use to run someone else's protein folding software etc. to design custom pathogens, and I really hope all the DNA printing companies have good multi-layer defences (all the way from KYC or similar to analysing what they've been asked to make and content-filtering it) by that point.

Post reply on HN