Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

521–530 of 936 posts

Re: I resigned from Anthropic today

#521

Post by pingou »

mrpopo wrote: post #495
computerphage wrote: post #472

Earlier quoted context omitted.

Why do you think global warming could kill literally every human being? What's the scenario in which that happens?

Runaway warming leading to a hot "supergreenhouse" climate was theorized. Alterbatively a snowpiercer scenario creating a snowball earth. Or, once billions of people die from climate change, planetary wars will start on the last hospitable areas, leading to the end of humanity. In contrast, AI ending "humanity" is limited by our own material power, would likely be stoppable, and would not easily reach areas isolated…

Why would it not easily reach areas isolated from technology? If a nefarious AI wins a war, it would be one of the easiest thing for it to locate any survivors anywhere on the planet.

Re: I resigned from Anthropic today

#522

Post by IsTom »

pingou wrote: post #497
King-Aaron wrote: post #477

Earlier quoted context omitted.

In the exact same way major sudden changes in the climate have lead to the extinction of the majority of plants an animals over this planets history. We are not special.

We are special in many ways, we have technology, we are everywhere, and we can think and plan (although considering global warming that is debatable). Global warming would have to create dramatic conditions everywhere on the planet, not leaving any small pocket of survivability to make humans extinct. Possible? Theoretically yes, but pretty unlikely. But obviously extreme global warming would be a catastrophe even if…

Technology depends on a lot of people cooperating around the globe. Disrupt a few critical chains (power generation, fertilizers, computers), add a bit of good old war and you'll soon be in the era before Haber process and a lot of people die. And then the remaining people will have trouble keeping that level of technology with how sudden the change would be.

Re: I resigned from Anthropic today

#523

Post by Ekaros »

petesergeant wrote: post #463
riffraff wrote: post #324

Earlier quoted context omitted.

Abstractly, yes but concretely, how? Many terrorist organizations would like to have a nuclear bomb, but don't.

> Abstractly, yes but concretely, how? AI has the capability to perform any function that can be performed over a network. You would hope that every system that can launch a nuke is properly and actually totally air-gapped, but there are lots of things you'd hope that turn out to not be true.

They are not totally. As such I would give non zero chance of even current AI to be able to socially manipulate a launch after sufficient hype up period... So prolonged campaign might make it possible.

Re: I resigned from Anthropic today

#524

Post by XorNot »

icepush wrote: post #423
thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Those are things that pose potential harm to great fractions of humanity (multiple billions of people) but none of them poses any threat to the actual extinction of all humanity.

Arguing whether "all" or just "most" of humans dying is certainly a choice.

Survival of the species isn't enough.

Re: I resigned from Anthropic today

#525

Post by andai »

foogazi wrote: post #233
aogaili wrote: post #122

HN crowed need to make up their minds.. Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots? Are they proofing or stealing math?

It doesn’t need to be all of them And it depends on the prompt

Do not annihilate mankind. (Make no mistakes!)

Re: I resigned from Anthropic today

#526

Post by Dylan16807 »

King-Aaron wrote: post #477
pingou wrote: post #471

Earlier quoted context omitted.

How is global warning gonna kill all of humanity? Even a full scale nuclear war wouldn't manage that. They could destroy most civilisations, culture and scientific achievements though.

In the exact same way major sudden changes in the climate have lead to the extinction of the majority of plants an animals over this planets history. We are not special.

We have a massive survival range and we're on every continent. That kind of sudden climate change is not nearly enough to wipe us out. Some other climate scenarios might. Nuclear war definitely could.

Re: I resigned from Anthropic today

#527

Post by nurumaik »

pingou wrote: post #471
thomascountz wrote: post #456

Earlier quoted context omitted.

This is not correct. Please go through each one again, taking special note of global warming and nuclear weapon development .

How is global warning gonna kill all of humanity? Even a full scale nuclear war wouldn't manage that. They could destroy most civilisations, culture and scientific achievements though.

Because nuclear war balances itself: more nuclear bombings = less nuclear weapons, less people to send them. It will naturally stop at some fraction of humanity left which is incapable of making and launching more nukes

Re: I resigned from Anthropic today

#528

Post by mofeien »

matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

How about sandbox escape + cyber security collapse + 50 (or 500) deadly and highly contagious novel pathogens with long incubation period that humans can't possibly roll out vaccines for simultaneously.

At least the first two should seem like a near-term worry after the past five months.

Re: I resigned from Anthropic today

#529

Post by khafra »

artemisart wrote: post #333
khafra wrote: post #322

Earlier quoted context omitted.

I appreciate your ability to separate sharing the belief itself from approval of acting on sincerely-held principle. However, I think the danger is much more plausible than you do. First, and least important, consider that self-replicating, solar-powered factories aren't magic; they're algae. Second, and more important, consider this fully non-magic route to doom: - We continue putting AI in charge of more things - I…

> First, and least important, consider that self-replicating, solar-powered factories aren't magic; they're algae. But what is that supposed to mean? Because humanity is not facing existential threat from algae.

That's why that was the least important objection, independent from the second, and only intended to address the "magic" claim, by showing that microscopic self-replicators already exist.

By analogy, consider how you might respond if someone claimed that there's no possible danger from pocket-sized projectile launchers, because they would require some magic means of propulsion that didn't depend on a taut string attached to a long, flexible arm:

You could reply that atlatls can launch projectiles without using a taut string. Atlatls are not pocket-sized, but they are sufficient to establish that projectiles can be non-magically launched without a full bow. You could then go on to describe a sling, or derringer; and these would not be invalidated by your initial objection to the "magic" part.

Re: I resigned from Anthropic today

#530

Post by jarek83 »

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Well, it is about LLMs and humans I believe. Don't forget that the first nuclear bomb tests were let go despite some of the scientists' concerns about possibility of dooming the world as they were not sure about all reactions that would happen.

With LLMs we don't even hesitate to call it black box while still pushing its capabilities.

Post reply on HN