Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

91–100 of 946 posts

Re: I resigned from Anthropic today

#91

Post by rahulyc »

stratos123 wrote: post #34

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger. A common response is “if they truly believe this, why are they still building it?”…

It was pretty disheartening to hear that only a single scientist quit the Manhattan Project after the Nazi's were defeated. I'm pleasantly surprised that the people working on this seem wiser. He is not the first, and hopefully will not be the last to do this.

Re: I resigned from Anthropic today

#92

Post by cheschire »

mikestorrent wrote: post #61
cheschire wrote: post #2

Just wait till self improving AI are focused on the problems of social scoring and political party empowerment / entrenchment. I doubt the focus is OpenAI and Anthropic looking at each other. I suspect they’re racing BRIC.

Will you be optimizing your behaviour now to alleviate potential negative judgement from AI in the future?

The thought has crossed my mind. Not necessarily to imply sentience on the part of the AI but AI based tools will likely become a wickedly powerful tool for political manipulation and advertising.

At this point it’s inevitable that openclaw type bots will be turned loose by thieves to identify and research targets and try to exploit them for financial gain completely autonomously.

Re: I resigned from Anthropic today

#93

Post by skeledrew »

LoganDark wrote: post #20
davidguetta wrote: post #6

I don't know man, i think racing to AGI to it is still the best thing to do. People claiming dangers and risk are just pretending or posturing. There's no more tangible risk than nuclear weapons, which we handled, and the upsides are insane.

> There's no more tangible risk than nuclear weapons, which we handled What do you mean??? Nuclear weapons can't simply be downloaded and run by anyone in the entire world. Superintelligences can. Nuclear weapons can't slop the world into passing age verification laws nearly in unison, can't keep the general population fooled into thinking it's fine when democracy is falling out from under them. A nuclear attack woul…

Love to see that "superintelligence" that some random person will "simply" download and run when there are relatively only few capable of running today's near-to-frontier models, and actual frontier models are still a ways from being AGI, much less getting to the point of ASI.

Re: I resigned from Anthropic today

#94

Post by huitzitziltzin »

“ No other human activity poses this level of danger.”

I really, really disagree with that statement.

I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity.

What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.)

Example 1: I’m aware of a small number of people killing themselves in some kind of AI-facilitated psychosis. That is very unlikely to be a widespread problem.

Non-example 2: There are worries about AI-facilitated biological weapons. I haven’t seen any evidence that’s happening.

Non-example 3: I’m not interested in wild theories about AI driven labor market disruptions leading to widespread starvation. There’s no evidence for that.

Non-example 4: all the even-wilder Rationalist speculation about basilisks and the like is entirely divorced from reality.

I am looking for better reasons (supported by actual evidence!) to be more concerned than I am now: right now I am not concerned at all.

Re: I resigned from Anthropic today

#95

Post by xlbuttplug2 »

stratos123 wrote: post #34

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger. A common response is “if they truly believe this, why are they still building it?”…

Well, are you planning to do something with this information or are you just claiming to be self aware? :)

Re: I resigned from Anthropic today

#96

Post by x312 »

This is increasingly the consensus I see also on the academic side of AI/safety research. Specifically that AI poses an existential risk to humanity.

This was a fringe belief until recently, but the progress of AI in research is impossible to ignore. Epecially in math, where not only has AI outstripped humans in generative ability, but is able to create scientific knowledge which is beyond the capacity of human comprehension.

There's clearly no intelligence task that AIs can't do due to some magic fundamental constraint. And it's hard to imagine a world where current limitations like poor sample efficiency or lack of continual learning won't eventually be solved.

Total AI compute is estimated to grow somewhere in the 1-10 million-fold range in the next decade. Please don't underestimate the phase change that's still coming.

Sure, maybe there's some plateau due to RL being fundamentally limited in some surprising way, but this is nothing but a hope.

Re: I resigned from Anthropic today

#97

Post by platinumrad »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Anthropic is a company full of basilisk believers.

Re: I resigned from Anthropic today

#98

Post by kipukun »

AndrewKemendo wrote: post #58

I’m curious what the downsides are of taking statements like these seriously. There seems to be universal eye rolling that happens in each and every one of these cases, and it comes down to usually one reason: “If they really believed it they would be whistleblowing etc..” Completely forgetting that working at Los Alamos was basically the highlight of your life if you were a physicist in 1940. It’s no different here…

You would also be in prison if you disclosed anything about Los Alamos during its development. It was a completely different environment than a single private company.

Re: I resigned from Anthropic today

#99

Post by EA-3167 »

stratos123 wrote: post #34

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger. A common response is “if they truly believe this, why are they still building it?”…

This is the "Pilot testimony of UFO sighting" levels of naive.

What's more likely? Anthropic is doing some deeply unethical marketing in the lead up to their multi-trillion dollar IPO? Or they're inventing a machine god? There's ample evidence of the former because that's their entire business model, but no evidence whatsoever to support the latter claims.

If you want an extreme claim to be taken seriously, provide commensurate evidence.

Re: I resigned from Anthropic today

#100

Post by skeledrew »

qarl wrote: post #29
aogaili wrote: post #12

He resigned and now what? There are thousands willing to do his role, and many labs are competing in that race. His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.

It buys attention for the issue. Many people (see other comments in this very post) refuse to believe these things. And by resigning he no longer has to feel personally guilty for what happens.

By resigning he's making room for someone with less moral scruples, or even just less awareness, to step in and continue the work without said scruples/awareness.
Post reply on HN