Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

Post reply on HN
Print view View original

81–90 of 716 posts

Re: Anthropic drops flagship safety pledge

#81

Post by baq »

TacticalCoder wrote: post #34
zer00eyz wrote: post #22

Earlier quoted context omitted.

> Then something went wrong, and no one knew how to stop it, This is the problem with every AI safety scenario like this. It has a level of detachment from reality that is frankly stark. If linesman stop showing up to work for a week, the power goes out. The US has show that people with "high powered" rifles can shut down the grid. We are far far away from a sort of world where turning AI off is a problem. There isnt…

> There isnt going to be a HAL or Terminator style situation ... I don't believe for a second we'll have an evil AI. However I do believe it's very likely we may rely on AI slop so much that we'll have countless outages with "nobody knowing how to turn the mediocrity off". The risk ain't "super-intelligent evil AI": the risk is idiots putting even more idiotic things in charge. And I'm no luddite: I use models daily.

> I don't believe for a second we'll have an evil AI.

Doesn’t have to be evil to be disastrous. Misaligned is plenty enough.

https://en.wikipedia.org/wiki/Instrumental_convergence

Re: Anthropic drops flagship safety pledge

#82

Post by ddxv »

Art9681 wrote: post #19

Of course the US is going to do this and of course its in Anthropics best interest to comply. Right now China is flooding HuggingFace with models that will inevitably have this capability. Right now there are hundreds of models being hosted that have been deliberately processed to remove refusals and their safety training. Everyone who keeps up with this knows about it. HF knows about it. And it is pretty obvious tha…

> Right now a kid can go download an Abliterated version of a capable open weight model and they can go wild with it.

Is the reason to ban or block free open weight models that you're worried what kids will do with them?

I'd imagine the economic case to be made is that the Western AI companies will ultimately not be able to compete with free open weight models. Additionally, open weight models will help to spread the economic gains by not letting a few monopolies capture them behind regulatory red tape.

Finally, I'd say the geopolitics angle of why open weight models are better is that if the West controls the open source software that will power it will be able to reap the benefits that soft power brings with it.

Re: Anthropic drops flagship safety pledge

#83

Post by nradov »

yunwal wrote: post #57
nradov wrote: post #46

Earlier quoted context omitted.

How is it a good thing to refuse to provide our warfighters with the tools that they need? I mean if we're going to have a military at all then we owe it to them to give them the best possible weapons systems that minimize friendly casualties. And let's not have any specious claims that LLMs are somehow special or uniquely dangerous: the US military has deployed operational fully autonomous weapons systems since the…

This is the US military we’re talking about so 95% of what they do is attacking people for oil. They don’t “need” more of anything, they’re funded to the tune of a trillion dollars a year, almost as much as every other military in the world combined. What holy mission do you think they’re going to carry out with the assistance of LLMs?

That's a total non sequitur. If you think the military is being tasked with the wrong missions, or too many missions, then take that up with the civilian political leadership. But it's not a valid reason to deny the warfighters the best possible weapons systems.

Personally I favor a less interventionist foreign policy. But that change can only come about through the political process, not by unaccountable corporate employees making arbitrary decisions about how certain products can be used.

Re: Anthropic drops flagship safety pledge

#84

Post by elric »

AndrewKemendo wrote: post #50
ashtonshears wrote: post #21

Earlier quoted context omitted.

The societal ills from collective tendancy to ignore red flags seems to be a human trait

It's in your nature to destroy yourselves

Defeatist bullshit becomes self-fulfilling at some point. "Oh we're all gonna die anyway so we might as well milk this thing for profit. Après moi la déluge."

Re: Anthropic drops flagship safety pledge

#86

Post by palmotea »

chris_money202 wrote: post #11

First they rushed a model to market without safety checks, and I said nothing. It wasn't my field. Then they ignored the researchers warning about what it could do, and I said nothing. It sounded like science fiction. Then they gave it control of things that matter, power grids, hospitals, weapons, and I said nothing. It seemed to be working fine. Then something went wrong, and no one knew how to stop it, no one had…

> First they rushed a model to market without safety checks, and I said nothing. It wasn't my field.

> Then they ignored the researchers warning about what it could do, and I...

...tried it and became an eager early adopter and evangelist. It sounded like something from a dystopian science function novel I enjoyed.

> Then [I] gave it control of things that matter, power grids, hospitals, weapons, and...

...my startup was doing well, and I was happy. We should be profitable next quarter.

> Then something went wrong, and no one knew how to stop it, no one had planned for it...

...and I was guilty as fuck,

FTFY, to fit the HN crowd.

Re: Anthropic drops flagship safety pledge

#87

Post by EagnaIonat »

tolmasky wrote: post #42

I don't understand how safety is taken seriously at all. To be clear, I'm not referring to skepticism that these companies can possibly resist the temptation to make unsafe models forever. No, I'm talking about something far more basic: the fact that for all the talk around safety, there is very little discussion about what exactly "safety" means or what constitutes "ethical" or "aligned" behavior. I've read reams of…

I would recommend reading up on the EU AI Act. It clearly defines what safety is in regards to the human race. Your questions are actually covered by it.

Re: Anthropic drops flagship safety pledge

#88

Post by ahtihn »

nradov wrote: post #83
yunwal wrote: post #57

Earlier quoted context omitted.

This is the US military we’re talking about so 95% of what they do is attacking people for oil. They don’t “need” more of anything, they’re funded to the tune of a trillion dollars a year, almost as much as every other military in the world combined. What holy mission do you think they’re going to carry out with the assistance of LLMs?

That's a total non sequitur. If you think the military is being tasked with the wrong missions, or too many missions, then take that up with the civilian political leadership. But it's not a valid reason to deny the warfighters the best possible weapons systems. Personally I favor a less interventionist foreign policy. But that change can only come about through the political process, not by unaccountable corporate e…

> But it's not a valid reason to deny the warfighters the best possible weapons systems.

Of course it is.

Think about it this way: if you could guarantee that the military suffers no human losses when attacking a foreign country, do you think that's going to more or less foreign interventions?

The tools available to the military influence policy, these things are linked.

US military is already overwhelmingly powerful, there's 0 reason to make it even more powerful.

Re: Anthropic drops flagship safety pledge

#90

Post by [deleted] »

nradov wrote: post #83
yunwal wrote: post #57

Earlier quoted context omitted.

This is the US military we’re talking about so 95% of what they do is attacking people for oil. They don’t “need” more of anything, they’re funded to the tune of a trillion dollars a year, almost as much as every other military in the world combined. What holy mission do you think they’re going to carry out with the assistance of LLMs?

That's a total non sequitur. If you think the military is being tasked with the wrong missions, or too many missions, then take that up with the civilian political leadership. But it's not a valid reason to deny the warfighters the best possible weapons systems. Personally I favor a less interventionist foreign policy. But that change can only come about through the political process, not by unaccountable corporate e…

[deleted]
Post reply on HN