Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gambling with our lives: AI researcher quits Anthropic with warning about safety

politico.eu

Post reply on HN
Print view View original

21–30 of 110 posts

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#21

Post by pjc50 »

glimshe wrote: post #5

What do people feel about this in China? Even if their models are well behind, they are not years behind. If we restrain US companies, assuming that is desirable, it would do nothing to deter China's and AI-pocalypse would come anyway in short notice.

China is a regulated discourse environment, but I get the impression that they're nowhere near as pessimistic.

Why would they be? Everything in China is under the control of the government. That includes the AI, all the telecoms infrastructure it might use, and all its power supplies.

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#23

Post by Jackpillar »

glimshe wrote: post #5

What do people feel about this in China? Even if their models are well behind, they are not years behind. If we restrain US companies, assuming that is desirable, it would do nothing to deter China's and AI-pocalypse would come anyway in short notice.

You act as if Chinas Ai labs exist in the same (non-existent) regulatory framework as US labs and that they're also helmed by a similar small gaggle of psychopathic egomaniacs who are richer than god. Have you considered that perhaps Chinas labs don't share the race to the bottom technological/economic death spiral?

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#26

Post by pluc »

dotancohen wrote: post #20
virgildotcodes wrote: post #6

At least we’ll eventually have an entity other than ourselves to blame for our annihilation.

No, the AI is still our responsibility.

Isn't it insane that even in the face of complete annihilation through one of our inventions, we go "that wasn't us"? We deserve that shit ten times over

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#27

Post by pjc50 »

dotancohen wrote: post #18
pjc50 wrote: post #14

Earlier quoted context omitted.

This is just a bizarre thing to say that you're working on technology with that high a downside potential. If you were saying that while running a biology lab, or building a nuclear reactor, people would be demanding your head on a spike. But by not quitting it's clear that he himself doesn't really believe it. Or rather, this shows the difference between "believe" (political) and "believe" (use as a basis for action…

Or he believes that other labs might get there first, and he is working to counter that threat. This is the Manhattan Project again.

How exactly does that work? The nuclear system of MAD relies on physical threat, lab A achieving ASI (artificial scary intelligence) does not prevent lab B achieving it.

I would like everyone involved to be a lot clearer about their threat models, with plausible series of clearly linked steps, rather than just sounding like a Vernor Vinge novel.

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#28

Post by meowkit »

pjc50 wrote: post #14
koolba wrote: post #4

> Evan Hubinger, Anthropic's staff lead on keeping the technology aligned with human goals and values, backed up Coxon claims in a follow-up post of his own, though he didn't quit the company. > "Jacob is correct here — we really do earnestly believe AI could kill all humans," he said. > Hubinger estimated the chances of that happening to be higher than ten percent within the next decade, and added that there's no pl…

This is just a bizarre thing to say that you're working on technology with that high a downside potential. If you were saying that while running a biology lab, or building a nuclear reactor, people would be demanding your head on a spike. But by not quitting it's clear that he himself doesn't really believe it. Or rather, this shows the difference between "believe" (political) and "believe" (use as a basis for action…

A simple explanation:

This is a highly uncertain and dangerous scenario. Breeding ground for anxiety.

High agency people often deal (cope) with anxiety by trying to control outcomes. Some just flee the situation altogether.

We have an example of both here: one employee leaves, one stays.

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#29

Post by jurgenburgen »

Bengalilol wrote: post #7

> Both OpenAI and Anthropic have recently flagged incidents in which agents powered by their models went rogue I may be biased and somewhat off topic, but I see these incidents as some of the most significant of the past century. I genuinely don't understand why these companies aren't taking a smarter approach to them. The latest analyses have been, at best, laughable: identify the vulnerability, patch it, and move o…

Ultimately these are unserious companies ran by unserious people. They don’t even have a business plan, why would they bother with some kind of sensible security policy?

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#30

Post by pjc50 »

MrThoughtful wrote: post #25

Why would AI wipe us out? We have not wiped out apes, ants, and most other species. We even have discussions about how to actively save them from extinction.

Ender's game model, presumably: the AI helpfully assists a human to build a nuclear bomb / pandemic virus / autonomous killer robot swarm in their basement. But again, I would like people to be clearer about how the threat is supposed to work rather than just making SF references.
Post reply on HN