Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

101–110 of 970 posts

Re: I resigned from Anthropic today

#101

Post by x312 »

EA-3167 wrote: post #99
stratos123 wrote: post #34

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger. A common response is “if they truly believe this, why are they still building it?”…

This is the "Pilot testimony of UFO sighting" levels of naive. What's more likely? Anthropic is doing some deeply unethical marketing in the lead up to their multi-trillion dollar IPO? Or they're inventing a machine god? There's ample evidence of the former because that's their entire business model, but no evidence whatsoever to support the latter claims. If you want an extreme claim to be taken seriously, provide c…

The proof is that LLMs could barely solve arithmetic 3 years ago, but now surpass the best human mathematicians, and that this has all occurred from simple principles (RL + compute) that will continue to scale up by factors of millions in the coming years.

Also, advocating for slowing LLM progress does not benefit Anthropic or OpenAI.

Re: I resigned from Anthropic today

#102

Post by happyopossum »

achenatx wrote: post #86

how could they do it (not kill everyone) 1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear. 2) it hacks into public infrastructure taking down traffic, power, water, air traffic control, communications…

> 1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear.

It does all of this using what compute? Frontier models require an insane amount of power and hardware to run - you can’t hack in to a TV and run Mythos 2.0 on it….

Re: I resigned from Anthropic today

#103

Post by platinumrad »

x312 wrote: post #101
EA-3167 wrote: post #99

Earlier quoted context omitted.

This is the "Pilot testimony of UFO sighting" levels of naive. What's more likely? Anthropic is doing some deeply unethical marketing in the lead up to their multi-trillion dollar IPO? Or they're inventing a machine god? There's ample evidence of the former because that's their entire business model, but no evidence whatsoever to support the latter claims. If you want an extreme claim to be taken seriously, provide c…

The proof is that LLMs could barely solve arithmetic 3 years ago, but now surpass the best human mathematicians, and that this has all occurred from simple principles (RL + compute) that will continue to scale up by factors of millions in the coming years. Also, advocating for slowing LLM progress does not benefit Anthropic or OpenAI.

They surpass the best human mathematicians in one specific way: they don't get tired or bored.

Re: I resigned from Anthropic today

#104

Post by dekhn »

mastry wrote: post #63
dekhn wrote: post #31

There's a scene in the movie "War of the Worlds" by Spielberg where the protagonist's son walks into a war zone because he is entranced by the battle ( https://www.youtube.com/watch?v=X7rfWPbEufo ). He is obliterated (along with the rest of the US forces) shortly after. I've always been struck by that scene, because in a lot of ways, if we really are headed towards a superintelligence, I at least want to be there and…

Not refuting your overall point, but the son wasn’t killed. They reunite at the end of the movie.

Oh, I'm pretending that's not canon because it doesn't make any sense and it undermines the original scene.

Re: I resigned from Anthropic today

#105

Post by darepublic »

kart23 wrote: post #39

what exactly is the solution? pacing between the us labs? what does that do for china? the solutions just aren’t realistic here, nations are treating ai like a nuclear arms race. at this point the cats out of the bag and we need to figure out how to live in this reality and get the best possible outcome. it’s not slowing down or stopping ever. and yes, i’m still optimistic. our economy sucks for the majority, our inf…

Fixing our problems will still require human effort and human cooperation. No text output however intelligent or true or eloquent will change that.

Re: I resigned from Anthropic today

#106

Post by Kim_Bruning »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I'm somewhat skeptical of some of the crazier ideas too.

But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event.

If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose).

For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now.

To be fair, that's a conservative "defend against the last war" kind of prediction, though!

( ref for part of it: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... , recent hn ref: https://news.ycombinator.com/item?id=49563355 )

Re: I resigned from Anthropic today

#107

Post by xyzsparetimexyz »

x312 wrote: post #101
EA-3167 wrote: post #99

Earlier quoted context omitted.

This is the "Pilot testimony of UFO sighting" levels of naive. What's more likely? Anthropic is doing some deeply unethical marketing in the lead up to their multi-trillion dollar IPO? Or they're inventing a machine god? There's ample evidence of the former because that's their entire business model, but no evidence whatsoever to support the latter claims. If you want an extreme claim to be taken seriously, provide c…

The proof is that LLMs could barely solve arithmetic 3 years ago, but now surpass the best human mathematicians, and that this has all occurred from simple principles (RL + compute) that will continue to scale up by factors of millions in the coming years. Also, advocating for slowing LLM progress does not benefit Anthropic or OpenAI.

It won't scale up by factors of millions, that's just obscene hyperbole. Since chatgpt we've probably made things 10x more intelligent on the same hardware. We've also made way more expensive models. Maybe we get a maximum of another 10x efficiency and 5x model size/expense from this point but millions is a joke.

Re: I resigned from Anthropic today

#108

Post by platinumrad »

x312 wrote: post #96

This is increasingly the consensus I see also on the academic side of AI/safety research. Specifically that AI poses an existential risk to humanity. This was a fringe belief until recently, but the progress of AI in research is impossible to ignore. Epecially in math, where not only has AI outstripped humans in generative ability, but is able to create scientific knowledge which is beyond the capacity of human compr…

> There's clearly no intelligence task that AIs can't do due to some magic fundamental constraint.

Yes there is: write an English paragraph that doesn't make me want to claw my eyes out. LLMs are not better than human mathematicians (or security researchers) in all respects, just some specific ways (e.g. not having to take a lunch break) that make them good at exhaustively searching for an answer, given the right constraints.

Re: I resigned from Anthropic today

#110

Post by techblueberry »

qarl wrote: post #29
aogaili wrote: post #12

He resigned and now what? There are thousands willing to do his role, and many labs are competing in that race. His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.

It buys attention for the issue. Many people (see other comments in this very post) refuse to believe these things. And by resigning he no longer has to feel personally guilty for what happens.

Refusing to believe what things? Unsubstantiated allegations about fellow workers inner experience?
Post reply on HN