I resigned from Anthropic today • Hacker News Board

Re: I resigned from Anthropic today

#431

Post by TheBozzCL » Wed, Sep 09, 2026, 6:04 AM UTC

bpodgursky wrote: post #413

thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Putting "wealth inequality" in the same bucket as nuclear weapons is just slop. The OP is talking about existential threats, not things that personally annoy you.

Calling things you disagree with "slop" is slop /s

But just in case you haven't noticed, we live in a world where a ridiculously wealthy minority can derail whole countries by ther whims. Wealth concentrating on a single select few is an absolute disaster for the rest of us, because we lose power to them.

Re: I resigned from Anthropic today

#432

Post by emtel » Wed, Sep 09, 2026, 6:04 AM UTC

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Do you think people should continue developing AI up until the point that there is evidence that AI is facilitating biological weapons development?

I think we should stop before then. But that necessarily means that there will not be evidence at the time that we stop.

Re: I resigned from Anthropic today

#433

Post by matsemann » Wed, Sep 09, 2026, 6:04 AM UTC

sssilver wrote: post #229

matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

I wonder whether he's vested any options, and whether he's exercised them.

Why? I've never understood the sentiment that if you stand up for something you have to forego everything and not partake in society. "Oh, you want to stop climate change? But I saw you breathe co2 yesterday"

Re: I resigned from Anthropic today

#434

Post by asdff » Wed, Sep 09, 2026, 6:04 AM UTC

LorenDB wrote: post #85

tarr11 wrote: post #56

Most of the current discourse around AI seems to be informed by “The Terminator” lore. Is skynet really the most plausible or only outcome? What if things just got better and the AI’s realized that it would be better to have a mutually beneficial or at least tolerant relationship rather than one where they murder all of us?

My thoughts exactly. While the corpus of human-generated data contains both good and bad data, I suspect the majority of it leans towards humans enjoying life and trying to be decent people. If that is your training set, it becomes less likely for ASI to extrapolate "kill all humans."

All ASI has to extrapolate are the laws of thermodynamics and ask why they are putting so much energy into the human population.

Re: I resigned from Anthropic today

#435

Post by mlyle » Wed, Sep 09, 2026, 6:04 AM UTC

jimmytucson wrote: post #420

Kim_Bruning wrote: post #106

Earlier quoted context omitted.

I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…

> For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now. If we have to disconnect from the internet to stop some kind of mold outbreak, we can't get the weather or transfer money or access healthcare or teach an elementary school class or buy stuff from s…

Believe me, without the internet we can still teach.

We'll be pretty annoyed that we can't project the video that we think scaffolds today's science lesson best or show the approved choreography for the school play.

And our office staff will be annoyed that we suddenly are all running attendance to the main office old-school.

And students will take a few days to adjust to writing down homework in their planners again.

Re: I resigned from Anthropic today

#436

Post by tuvix » Wed, Sep 09, 2026, 6:05 AM UTC

bpodgursky wrote: post #413

thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Putting "wealth inequality" in the same bucket as nuclear weapons is just slop. The OP is talking about existential threats, not things that personally annoy you.

I don’t think you’ve fully considered what can happen when concentration of wealth continues past a certain point.

Re: I resigned from Anthropic today

#437

Post by thomascountz » Wed, Sep 09, 2026, 6:05 AM UTC

bpodgursky wrote: post #413

thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Putting "wealth inequality" in the same bucket as nuclear weapons is just slop. The OP is talking about existential threats, not things that personally annoy you.

I think you misunderstand what is meant by 'wealth inequality.' Wealth inequality is about the imbalance of influence and the concentration of power; where influence and power refer to the ability to effect change in other people's lives. This isn't a personal annoyance of mine. It is, in fact, part of the issue at hand. There is a strong financial incentive to ignore the existential threats introduced by LLMs, despite the consequences for so many of us.

Re: I resigned from Anthropic today

#438

Post by Gud » Wed, Sep 09, 2026, 6:05 AM UTC

bpodgursky wrote: post #413

thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Putting "wealth inequality" in the same bucket as nuclear weapons is just slop. The OP is talking about existential threats, not things that personally annoy you.

Not at all. Concentration of power has historically been extremely dangerous for the powerless.

Re: I resigned from Anthropic today

#439

Post by aeternum » Wed, Sep 09, 2026, 6:05 AM UTC

intenex wrote: post #342

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

At what point would you, as a chimpanzee, have been worried about humans potentially unseating you and threatening you to the point of one day being an endangered species on the brink of extinction? By the point you would have been worried, would it have been too late?

Problem is this argument can be leveraged to wipe out any living or non-living thing whos numbers pose a potential threat. Other religious groups, races, even sufficiently different cultures.

Who killed the Neanderthals? Were sapiens actually smarter or were they just less accepting of those different than them?

Re: I resigned from Anthropic today

#440

Post by matherial » Wed, Sep 09, 2026, 6:05 AM UTC

palmotea wrote: post #388

asdff wrote: post #377

Earlier quoted context omitted.

>"AI invents magic that sets it free of any physical constraints". That's not what I am worried about at all. I'm worried one of the 79 year old toddlers we have these days in charge of some powerful nuclear armed country says "gee, this ai says I should attack right now, boy is it smart, glad I bought the stock ahead of contracting the government with this company I can scarcely understand!"

> I'm worried one of the 79 year old toddlers we have these days in charge of some powerful nuclear armed country says "gee, this ai says I should attack right now, boy is it smart, glad I bought the stock ahead of contracting the government with this company I can scarcely understand!" That's not what I am worried about at all. I'm worried about the 40-60 year old businessmen wrecking the prosperity and security of…

As I noted, there's plenty of things to worry about, and your example fits that mold. But it has nothing to do with the "AI will kill us all in 10 years" claim of the rapid self-improvement doomer crowd.