Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

731–740 of 1001 posts

Re: I resigned from Anthropic today

#731

Post by thomascountz »

chrisjj wrote: post #648
thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

> See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. The difference is that none of those are available to individuals.

Not to be completely pedantic, but the fact that one, and only one, individual in the United States of America can deploy any of their 5000+ nuclear weapons undermines this argument. Furthermore, LLMs (as they are today) which pose the treats mentioned in TFA, are expressly not available to individuals (for the time being). However, my point was never about individuals, but rather of systematic incentive structures which actively make pathways to specific harms possible—if not probable.

Re: I resigned from Anthropic today

#732

Post by Yizahi »

Perseids wrote: post #636
thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Let me argue on a technicality first: None of these are extinction level events. If global warming disrupts 99% of all crop production, the remaining 1% is still plenty enough to sustain a stable, if miserable, population. In fact you just need about 5k people for a stable gene pool[1]. Of the classical threats, only bioweapons got a shot at extinction, but even that is hard, given the (few) remaining truly secluded…

You are confusing humanity and "humanity". Humanity-species is indeed rather hard to exterminate. Now if we are talking about actual individual humans, then 95% death rate is quite literally The Extinction.

This reminds me how people are misunderstanding and incorrectly quoting George Carlin sketch. Sure, the "Earth" will be fine. As in - the ball of rock will be fine. But we are not thinking about rocks when saying "Earth is in danger".

I'm pretty sure there is a formal name for this kind of semantic and pedantic substitution.

Re: I resigned from Anthropic today

#733

Post by hegelstoleit »

skeledrew wrote: post #148
hegelstoleit wrote: post #119

Earlier quoted context omitted.

That's not necessarily true, and you can use that argument to justify doing any immoral job. Just because someone else might be willing to do it isn't a reason to continue doing it.

> not necessarily true It's a possibility that is increased by their action. One leaves, a space is now open that will likely eventually be filled. And the chance of someone with equal/higher scruples filling it is very slim (unless you somehow know that the good amount of those who qualify and apply for the position have equal/higher scruples). That's just logic and math.

You missed the point.

> Just because someone else might be willing to do it isn't a reason to continue doing it.

Hence, someone filling the position after you leave isn't a reason to continue doing it, whereas the fact that it's an immoral thing to do is a reason to stop doing it.

Someone filling the position after you leave has no bearing on the reasons that are relevant to what you ought to do here. It's like deciding to not buy a ticket to see a movie because someone else is going to buy the ticket anyways even if you don't purchase the ticket. It has no relevance to whether you should watch the movie or not, just as someone else taking the job has no relevance to whether you ought to do the job.

Re: I resigned from Anthropic today

#734

Post by AdamN »

sharts wrote: post #726
Aeolun wrote: post #687

Earlier quoted context omitted.

That's basically the reasoning behind MAD, and it checks out? See what happened to any nation that ever gave away their nukes.

Shouldn’t everyone have nukes then, for maximum peace?

That's been proposed before btw and game theorists can justify that as being more stable in some ways. However it reduces the power of the incumbents (so why let it happen?) and the odds of having an irrational actor with nukes goes up.

Re: I resigned from Anthropic today

#735

Post by thisoneisreal »

singularity2001 wrote: post #701
eloisant wrote: post #685

Earlier quoted context omitted.

I still have to read a compelling argument on how AI will "extinct" humanity.

There are future scenarios in which swarms of drones hunt down every single one of us, but why would they? And currently it makes absolutely zero sense because they are completely dependent on us. And even if not, it would be like humanity going on a mission to kill every single cat on Earth. It makes zero sense.

I'm not convinced by the doomsday scenarios either, but I think there's a keyword in your post: "sense." These things don't have "sense." They do nonsensical things all the time, often almost immediately when given a task. So I think the main risk is letting them run wild in this digital world we created to precede them. Too much important stuff is wired up to computers, and we're giving them incredible access to command those computers.

Re: I resigned from Anthropic today

#736

Post by sethammons »

shafyy wrote: post #704
rossant wrote: post #697

Earlier quoted context omitted.

An AI, either acting autonomously or under human direction, hacks Russian/North Korea/etc. intelligence systems and convinces them that the US has launched ballistic missiles at them. The end.

Watched one too many second-tier disaster movies?

We solved this one: have it play tic tac toe against itself. Checkmate AI

Re: I resigned from Anthropic today

#737

Post by frankacter »

rossant wrote: post #697
thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

An AI, either acting autonomously or under human direction, hacks Russian/North Korea/etc. intelligence systems and convinces them that the US has launched ballistic missiles at them. The end.

A strange game. The only winning move is not to play.

Re: I resigned from Anthropic today

#738

Post by blueblisters »

eloisant wrote: post #685
Perseids wrote: post #636

Earlier quoted context omitted.

Let me argue on a technicality first: None of these are extinction level events. If global warming disrupts 99% of all crop production, the remaining 1% is still plenty enough to sustain a stable, if miserable, population. In fact you just need about 5k people for a stable gene pool[1]. Of the classical threats, only bioweapons got a shot at extinction, but even that is hard, given the (few) remaining truly secluded…

I still have to read a compelling argument on how AI will "extinct" humanity.

The “how” is pretty hand wavy and rationalists/safety-ists usually say we probably don’t have the capacity to reason about that.

But the “why” is pretty convincing imo.

Long horizon alignment is obviously very hard and it’s not inconceivable that models optimized with underspecified goals converge to a conclusion that they need to hoard resources (instrumental convergence regardless of the terminal goal).

At that point a sufficiently capable model might view humanity like we do animals - worth preserving but not if we impede the model's goals.

Re: I resigned from Anthropic today

#739

Post by chrisjj »

aogaili wrote: post #122

HN crowed need to make up their minds.. Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots? Are they proofing or stealing math?

> Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots?

Both? Do not underestimate the power of a parrot given enough processing power and data.

Re: I resigned from Anthropic today

#740

Post by philipallstar »

sharts wrote: post #726
Aeolun wrote: post #687

Earlier quoted context omitted.

That's basically the reasoning behind MAD, and it checks out? See what happened to any nation that ever gave away their nukes.

Shouldn’t everyone have nukes then, for maximum peace?

You want people with a lot to lose in a nuclear war to have nuclear weapons, and to make sure no-one else does.
Post reply on HN