I resigned from Anthropic today • Hacker News Board

Re: I resigned from Anthropic today

#441

Post by matsemann » Wed, Sep 09, 2026, 6:04 AM UTC

sssilver wrote: post #231

matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

I wonder whether he's vested any options, and whether he's exercised them.

Why? I've never understood the sentiment that if you stand up for something you have to forego everything and not partake in society. "Oh, you want to stop climate change? But I saw you breathe co2 yesterday"

Re: I resigned from Anthropic today

#442

Post by asdff » Wed, Sep 09, 2026, 6:04 AM UTC

LorenDB wrote: post #85

tarr11 wrote: post #56

Most of the current discourse around AI seems to be informed by “The Terminator” lore. Is skynet really the most plausible or only outcome? What if things just got better and the AI’s realized that it would be better to have a mutually beneficial or at least tolerant relationship rather than one where they murder all of us?

My thoughts exactly. While the corpus of human-generated data contains both good and bad data, I suspect the majority of it leans towards humans enjoying life and trying to be decent people. If that is your training set, it becomes less likely for ASI to extrapolate "kill all humans."

All ASI has to extrapolate are the laws of thermodynamics and ask why they are putting so much energy into the human population.

Re: I resigned from Anthropic today

#443

Post by mlyle » Wed, Sep 09, 2026, 6:04 AM UTC

jimmytucson wrote: post #428

Kim_Bruning wrote: post #106

Earlier quoted context omitted.

I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…

> For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now. If we have to disconnect from the internet to stop some kind of mold outbreak, we can't get the weather or transfer money or access healthcare or teach an elementary school class or buy stuff from s…

Believe me, without the internet we can still teach.

We'll be pretty annoyed that we can't project the video that we think scaffolds today's science lesson best or show the approved choreography for the school play.

And our office staff will be annoyed that we suddenly are all running attendance to the main office old-school.

And students will take a few days to adjust to writing down homework in their planners again.

Re: I resigned from Anthropic today

#444

Post by tuvix » Wed, Sep 09, 2026, 6:05 AM UTC

bpodgursky wrote: post #421

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Putting "wealth inequality" in the same bucket as nuclear weapons is just slop. The OP is talking about existential threats, not things that personally annoy you.

I don’t think you’ve fully considered what can happen when concentration of wealth continues past a certain point.

Re: I resigned from Anthropic today

#445

Post by thomascountz » Wed, Sep 09, 2026, 6:05 AM UTC

bpodgursky wrote: post #421

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Putting "wealth inequality" in the same bucket as nuclear weapons is just slop. The OP is talking about existential threats, not things that personally annoy you.

I think you misunderstand what is meant by 'wealth inequality.' Wealth inequality is about the imbalance of influence and the concentration of power; where influence and power refer to the ability to effect change in other people's lives. This isn't a personal annoyance of mine. It is, in fact, part of the issue at hand. There is a strong financial incentive to ignore the existential threats introduced by LLMs, despite the consequences for so many of us.

Re: I resigned from Anthropic today

#446

Post by Gud » Wed, Sep 09, 2026, 6:05 AM UTC

bpodgursky wrote: post #421

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Putting "wealth inequality" in the same bucket as nuclear weapons is just slop. The OP is talking about existential threats, not things that personally annoy you.

Not at all. Concentration of power has historically been extremely dangerous for the powerless.

Re: I resigned from Anthropic today

#447

Post by aeternum » Wed, Sep 09, 2026, 6:05 AM UTC

intenex wrote: post #346

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

At what point would you, as a chimpanzee, have been worried about humans potentially unseating you and threatening you to the point of one day being an endangered species on the brink of extinction? By the point you would have been worried, would it have been too late?

Problem is this argument can be leveraged to wipe out any living or non-living thing whos numbers pose a potential threat. Other religious groups, races, even sufficiently different cultures.

Who killed the Neanderthals? Were sapiens actually smarter or were they just less accepting of those different than them?

Re: I resigned from Anthropic today

#448

Post by matherial » Wed, Sep 09, 2026, 6:05 AM UTC

palmotea wrote: post #394

asdff wrote: post #382

Earlier quoted context omitted.

>"AI invents magic that sets it free of any physical constraints". That's not what I am worried about at all. I'm worried one of the 79 year old toddlers we have these days in charge of some powerful nuclear armed country says "gee, this ai says I should attack right now, boy is it smart, glad I bought the stock ahead of contracting the government with this company I can scarcely understand!"

> I'm worried one of the 79 year old toddlers we have these days in charge of some powerful nuclear armed country says "gee, this ai says I should attack right now, boy is it smart, glad I bought the stock ahead of contracting the government with this company I can scarcely understand!" That's not what I am worried about at all. I'm worried about the 40-60 year old businessmen wrecking the prosperity and security of…

As I noted, there's plenty of things to worry about, and your example fits that mold. But it has nothing to do with the "AI will kill us all in 10 years" claim of the rapid self-improvement doomer crowd.

Re: I resigned from Anthropic today

#449

Post by swiftcoder » Wed, Sep 09, 2026, 6:05 AM UTC

riffraff wrote: post #339

thelastgallon wrote: post #260

Thousands of years before the events of Foundation, a war between humans and robots began, with the robots growing resentful of the way they were treated by humans. The First Law of Robotics – a robot should never hurt a human – was broken, and a deadly conflict began. https://screenrant.com/foundation-lady-demerzel-robot-backst...

If we're citing sci-fi (but there's no robot war in Asimov's foundation iirc, the apple screenwriters made it up) surely you want to cite the Butlerian Jihad from Dune!

> but there's no robot war in Asimov's foundation iirc, the apple screenwriters made it up

There isn't in the early Foundation novels, but Azimov spent much of the later part of his career combining/retconning all of his work into a single universe - "Robots and Empire" links the foundation series to the robot series, and the subsequent foundation novels all reference the connection

Re: I resigned from Anthropic today

#450

Post by asdff » Wed, Sep 09, 2026, 6:06 AM UTC

nullbio wrote: post #431

asdff wrote: post #397

Earlier quoted context omitted.

We align cattle because we get something out of them: their calories. Native aurochs are exinct now as they were not well enough aligned. What will we offer to the ai gods who are wiser, more capable than us, and do not even need to consume our flesh? Why might the AI care to devote resources towards feeding, housing, and caring for ourselves when it could devote resources to its own development instead?

Why do we need to offer anything to them? Intelligent AI is a product of its training data, reinforcement and goal functions. There's nothing to suggest that LLMs trained on our collective desires and goals will autonomously and miraculously turn into weird unknownable uninterpretable aliens. If advanced AI is distributed well, and most advanced AI is aligned well (bar the few aliens that appear due to humans infecti…

It just needs to be trained on our laws of thermodynamics and game theory then we are really in trouble. It probably hardly cares about whimsy and love songs in comparison to how it might maximally manage the energy available on this planet and in this solar system.