Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic researcher says more than 10% chance AI "could kill all humans"

cbsnews.com

Post reply on HN
Print view View original

51–60 of 111 posts

Re: Anthropic researcher says more than 10% chance AI "could kill all humans"

#51

Post by embedding-shape »

ultrarunner wrote: post #36
embedding-shape wrote: post #12

Earlier quoted context omitted.

What... I think you might be sharing more about your internal psyche here than providing some generalized commentary on this story. Why would you need to bomb a chip fab because you think AI might lead to people getting killed in the future? I'm convinced of many things killing humans, yet I don't have any desire to bomb or kill others, I think this is pretty common, but who knows....

You’re likely taking this too literally. OP is saying that beliefs inspire action, but as of now we haven’t seen action from these people that support such an extreme belief. What we have seen is incredible hype (justified or not), so it’s more likely just a continuation of that.

> OP is saying that beliefs inspire action, but as of now we haven’t seen action from these people that support such an extreme belief.

That something is harmful to humans means you need to take extreme action? The person is already leaving a job, probably a well-paid one, doing something they generally liked, until they saw a different future. That is the "action" you apparently haven't seen yet.

Not sure how it's reasonable to expect everyone who believe that AI might be involved in killing people in the future, must mean you should become a terrorist essentially. Not everyone is trying to be a hero in their life, some just want to live it out until it ends, trying to survive until then.

Re: Anthropic researcher says more than 10% chance AI "could kill all humans"

#52

Post by testaccount28 »

mstank wrote: post #13

I’ve yet to see a rational argument for how we go from super intelligent LLMs to human extinction or extermination. I understand that some smart people are worried about it. I just haven’t come across a believable or understandable argument.

the optimizer controls the future. it doesn't kill humans directly, it just outcompetes them for all resources, including livable human environment.

Re: Anthropic researcher says more than 10% chance AI "could kill all humans"

#53

Post by madaxe_again »

kortilla wrote: post #32
VagabundoP wrote: post #10

If it does it will be at the hands of another human. AI to me is like the Nuclear race again. Super powers will be using it as a super weapon. I don't think AGI will wipe us out by itself.

I’m not so sure on this, the hugging face incident showed that a relatively benign task can make it do quite destructive things in the name of optimizing a relatively benign goal.

I think we need to worry about paperclip maximisation as much as we need to worry about “Jon is having a bad day so brews up a novel plague”. There are likely ways it can go wrong that we haven’t even considered, too.

Re: Anthropic researcher says more than 10% chance AI "could kill all humans"

#54

Post by lenerdenator »

The risks of AI, while mostly hypothetical, have been understood for years.

If they were really this concerned about the risks of AI re: the survival of the human race, they wouldn't have joined a company working in such a space to begin with.

I hate to be this cynical, but part of me wonders what the financial angle is here.

You have technically brilliant people working in a white-hot target for investment and who can draw a high salary. Anthropic is a hyper-scaler that has the moat of high hardware prices and very little else. Every day, that moat gets a little smaller, and FLOSS models get a little better at being "good enough" for the price. You're an Anthropic employee looking to jump on the next big wave since the wave Anthropic is riding is starting to peter out. You go on the record across the trades and news sites talking about how dangerous AI is. That creates a need for someone to make it less dangerous. In theory, you could satisfy that need, for the right amount of money.

Again, I hate to be this cynical, but... it's the tech industry.

Re: Anthropic researcher says more than 10% chance AI "could kill all humans"

#55

Post by mchinen »

The early batch of Anthropic employees were mostly rationalist-adjacent AI safety folk that were almost uniformly claiming P_DOOM > .10 three years ago, so I believe them to be earnest.

It's very interesting to me that besides the other small safety labs that don't actually produce frontier models, Anthropic manages to keep such a good reputation within that subculture compared to OpenAI. Despite having as crazy internal politics as OpenAI, they have converged quite a bit from the original vision of safety first through Darwinistic pressures.

At least, it seems this way from the outside. I'm curious if the view from the inside is that different.

edit: to be clear, my reading as an outsider is that Anthropic is seen as relatively better in the AI safety community, but has definitely dropped in absolute reputation too. This recent thread and the references show some of that: https://www.lesswrong.com/posts/6j3kBHdowGLCeqobg/dear-god-p...

Re: Anthropic researcher says more than 10% chance AI "could kill all humans"

#56

Post by gozucito »

I remember the guy who said a Google LLM model (Pre-chatGPT 3.5) was conscious before being fired.

This reminds me of that. That 10% number was just an ass pull since no one actually knows with any degree of certainty what lies ahead.

Humans have much more than a 50% chance of killing all humans from the looks of it, based on reactions to Global Warming, Covid, and anti-science grifters gone wild.

AI can help cure disease. That is 100%. And for that alone, slowing down and missing out on thousands of cures would condemn tens of millions, perhaps hundreds of millions to suffer and die needlessly.

Re: Anthropic researcher says more than 10% chance AI "could kill all humans"

#57

Post by Miner49er »

bwfan123 wrote: post #46

> 10% chance AI "could kill all humans" Why not 10% chance that it will create enormous prosperity for all ? This is why the average person is increasing pissed at AI in general. That it gets associated with negativity.

Well obviously that's why they still pursue it. That's a major chunk of the other 90%.

Re: Anthropic researcher says more than 10% chance AI "could kill all humans"

#59

Post by embedding-shape »

kortilla wrote: post #45
embedding-shape wrote: post #12

Earlier quoted context omitted.

What... I think you might be sharing more about your internal psyche here than providing some generalized commentary on this story. Why would you need to bomb a chip fab because you think AI might lead to people getting killed in the future? I'm convinced of many things killing humans, yet I don't have any desire to bomb or kill others, I think this is pretty common, but who knows....

Not “people getting killed”, this is “all humans”. The only other ELE for humans that could possibly be caused by other humans that has been feasible is nuclear war. And lots of blood has been shed to prevent new players from getting bombs.

Personally I'd wager humanity has a greater chance than 10% of us all dying in a huge nuclear war sometime in the future. Is the claim really that this belief cannot be taken seriously unless I somehow violently attack nuclear silos, centrifuges and similar?

Re: Anthropic researcher says more than 10% chance AI "could kill all humans"

#60

Post by ricardobayes »

Skyy93 wrote: post #9
benenrjdnz wrote: post #3

Probably for the best. Humanity had a good run, but this is no longer our civilization. Your best course of action is to start ripping the copper out of the walls.

Why the doomerism? In general humanity is living its best life compared to nearly every other step in time.

I personally struggle to see ways my life has meaningfully improved since 2019 or so, and I feel like many people share this view. I have been an unquestionable fanatic of tech startups back then, which I consider perhaps the "golden age".

These days I view tech startups with suspicion, I question what is the ulterior motive. Somehow when upstart companies were like "Pebble", this wasn't a thing.

Post reply on HN