Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

901–910 of 995 posts

Re: I resigned from Anthropic today

#901

Post by skrellm »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

> I really, really disagree with that statement.

But you're wrong, although not for the reasons you think (suggested by your examples).

> I don’t think ai models come close to nuclear weapons

Unless AI models are controlling the drones/icbms carrying those. https://www.theguardian.com/technology/2026/aug/18/autonomou...

> or to run-of-the-mill, everyday carbon emissions

Unless AI models are responsible for those said emissions. https://arstechnica.com/tech-policy/2026/08/amazon-funds-big...

> What’s the most dangerous thing that’s happened with an LLM so far?

That it's built solely on "trust me, bro"?

> right now I am not concerned at all.

You should be. There's no scenario where this ends well: if AI labs succeed, we all will loose our jobs. If they fail, then the bubble bursts. Economic crisis is inevitable either way, and that's just the tip of the iceberg.

Re: I resigned from Anthropic today

#902

Post by eudamoniac »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

There's no evidence building the torment nexus will be bad, actually! It's just speculation!

Re: I resigned from Anthropic today

#903

Post by bdelmas »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

“ No other human activity poses this level of danger.” Yet. But it may be and very soon.

Re: I resigned from Anthropic today

#904

Post by skeledrew »

hegelstoleit wrote: post #756
skeledrew wrote: post #148

Earlier quoted context omitted.

> not necessarily true It's a possibility that is increased by their action. One leaves, a space is now open that will likely eventually be filled. And the chance of someone with equal/higher scruples filling it is very slim (unless you somehow know that the good amount of those who qualify and apply for the position have equal/higher scruples). That's just logic and math.

You missed the point. > Just because someone else might be willing to do it isn't a reason to continue doing it. Hence, someone filling the position after you leave isn't a reason to continue doing it, whereas the fact that it's an immoral thing to do is a reason to stop doing it. Someone filling the position after you leave has no bearing on the reasons that are relevant to what you ought to do here. It's like decid…

> the fact that it's an immoral thing to do

This is carrying a lot of weight. What makes working on frontier models "immoral"? Because there is potential for misuse? Would you also say working at a knife factory is immoral? Or even at a nuclear or biological pathogens research lab? After all those too can be misused.

And if one does suspect a workplace is moving in a bad direction, and there's no immediate danger to them, is leaving and then talking about it really the responsible move? Like if you're on your way to meet friends late one evening and you see a small child alone and crying in an alleyway and a bunch of sleazy-looking guys hanging around, do you continue on to your meeting where you then tell your friends about the crying child you passed in an alley and what those guys may or may not do to the child?

Re: I resigned from Anthropic today

#905

Post by vonneumannstan »

davidguetta wrote: post #817
vonneumannstan wrote: post #18

Earlier quoted context omitted.

>People claiming dangers and risk are just pretending or posturing. I believe you are mentally ill. >There's no more tangible risk than nuclear weapons, which we handled Lol way to rewrite history. Nuclear armageddon is still a significant risk...

I am not rewriting. I am saying AI is not more dangerous in my opinion that Nuclear Weapons.

You are wrong. A teenager with enough GPUs could never dream of making a nuke but they are absolutely able to run capable AIs in their moms basement. Totally different class of danger

Re: I resigned from Anthropic today

#906

Post by Timon3 »

DangitBobby wrote: post #869
SupremumLimit wrote: post #798

Earlier quoted context omitted.

"value creation for one's customers" - that is a "not even wrong" level of misunderstanding of the system dynamics of capitalism. The primary driver is generation of _profit_ for owners of the means of production. Many things fall out of that - the tendency to monopolise, for example.

It really doesn't have to be that way. The problem boils down to, as with practically all human systems, is human behavior. There is no system you can devise that is run by humans and not corruptible and exploitable by the worst of us. Or even just everyday people.

First: How do you know that every system is corruptible and exploitable?

And second: What if other systems are less corruptible and exploitable? How do you know whether all systems have the same potential for corruption and exploitation?

Capitalism is a system that explicitly rewards greed, and that gives the most power to the people with the most to gain from exploiting said power. Why is it unreasonable to believe that other systems are likely better on these fronts?

Re: I resigned from Anthropic today

#907

Post by barnabee »

overtone1000 wrote: post #215
onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

I can't help but think the most plausible scenarios are the ones that have a little less machine supremacy and a little more human stupidity. The Matrix is less plausible than WarGames.

This

Already powerful and wealthy organisations/institutions/people leveraging somewhat powerful AI seem far more dangerous to me than the "AI decides to kill everyone" scenarios.

And therein is my problem: OpenAI and Anthropic are organisations like that, run by people like that, and they operate in collusion with others also fitting that description. The call is coming from inside the house.

The most likely way that AI creates a disaster is due to the ambition, greed, hubris, and naivety of these people who think they understand the risks[0] better than anyone else. I firmly believe they do not.

The first step to managing these risks is for the "frontier labs" to be willing to not just collaborate but meaningfully give up some of their power and control.

[0] by which I mean not just thinking about how AI might develop but also what's physically possible, how the AI might interact with the world, and how people/societies/organisations/governments might react and respond.

Re: I resigned from Anthropic today

#908

Post by danny_codes »

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

I disagree, it’s about computers.

The risk is that we lose control. This has been a staple of SciFi since forever. This guy has apparently seen enough that he thinks that’s a viable threat

Re: I resigned from Anthropic today

#909

Post by sambuccid »

fallingbananna wrote: post #858
sambuccid wrote: post #853

Earlier quoted context omitted.

There is a great book about this, published in 2014, called "superintelligence" by Nick Bostrom, i found it very serious and detailed. (It also felt like reading 100 sci-fy books in one)

Can you elaborate on that? What points was the book making?

I'll be honest I read it a long time ago, and not even in its entirety, so I don't remember exactly all the different ways it suggested AI could take over, but I remember being convinced it was definitively a possibility, and that there were many ways in which this could happen. What I remember clearly is how surprised I was to see these(what felt at the time) unlikely futures being expressed in such clear detail and with so many references to existing literature, just to give an idea there are 56 pages of small-font "footnotes".

Also just to clarify, the book talked about different types of AI, was not focused just on LLMs.

Re: I resigned from Anthropic today

#910

Post by senordevnyc »

copperx wrote: post #153
matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

They literally have never promised anything like that. They have explicitly said repeatedly that they want regulation, oversight, nationalization, a global slowdown, etc. And then I have to read page after page of cynical comments about how that's all just hype and marketing and them wanting regulation to block their competition and blah blah blah.

You people would only be happy if they stopped all AI development, but you literally cannot do that in an arms race and survive. I cannot fathom what is so difficult to understand about this.

Post reply on HN