Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

211–220 of 987 posts

Re: I resigned from Anthropic today

#211

Post by aesthesia »

jeffrwells wrote: post #159

I doubt this is a real person. Screams of propaganda. Sama saying GPT-2 is too dangerous to release…all over again. He joins Twitter for first time in 2026 with a nonsensical username unrelated to his real name, and follows 14 people but is somehow embedded in tech enough to work at Anthropic. I haven’t used twitter since 2014 and even I follow more people. His morals tell him to walk away from tens of millions in un…

@hilbertspaess is not a nonsensical user name. The accounts he follows are totally reasonable for an AI researcher. I think it's extremely believable that he created an account in January, followed a few people as part of the initial setup flow, and then forgot about it until now.

Re: I resigned from Anthropic today

#212

Post by underyx »

brookst wrote: post #183
underyx wrote: post #181

Earlier quoted context omitted.

"So you think in 3 years AI is going to solve longstanding math problems because it was used to write some coherent sentences?" — people with the same amount of foresight in 2023

Are you saying at anything that can solve longstanding math problems necessarily has the means, motive, and capability to kill 8 billion people in just 3 years?

No, the highest probability estimate I've seen in this thread is 10% chance in the next 3 years.

Re: I resigned from Anthropic today

#213

Post by threatofrain »

areoform wrote: post #206
nunez wrote: post #196

Earlier quoted context omitted.

Oh man, it is almost too easy to imagine how deadly a jailbroken Mythos-class open-weights model can be if in the wrong hands. The big labs scrape LITERALLY EVEYTHING and get fresh data from their users. Both of the big labs have massive contracts with defense agencies. If the open-weights models are just distillations of FMs...

How would it be lethal? Please specify. What would that theoretical entity be able to do that hasn't been done many, many times before?

Before we discussed how important security was, we got insurance, we made libraries and products, we used compliance software, etc. Except how honest were we about all that stuff? How much risk was actually in the air, and what was keeping us accountable on security in either direction of over or under-investment?

Now a reckoning is here. The potential to be attacked might actually translate to being attacked.

Re: I resigned from Anthropic today

#214

Post by foogazi »

slicktux wrote: post #201
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.

> They do as they are told

1. What about hallucinations ?

2. What are they told to do ?

Re: I resigned from Anthropic today

#215

Post by overtone1000 »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

I can't help but think the most plausible scenarios are the ones that have a little less machine supremacy and a little more human stupidity. The Matrix is less plausible than WarGames.

Re: I resigned from Anthropic today

#216

Post by ls612 »

adrithmetiqa wrote: post #164
ls612 wrote: post #147

Earlier quoted context omitted.

Despite all of your hyping up of the Huggingface incident it ultimately caused zero actual damage.

If two airplane manufacturers were found to have massive safety issues which nearly led to enormous fatalities (but no one actually died), would you be calling for them to ground their aircraft until safety was made the number one priority?

Historically it almost always takes actual fatalities rather than near misses to ground an aircraft, and aviation is famous for its obsession with safety compared to other industries.

Re: I resigned from Anthropic today

#217

Post by reasonableklout »

tarr11 wrote: post #56

Most of the current discourse around AI seems to be informed by “The Terminator” lore. Is skynet really the most plausible or only outcome? What if things just got better and the AI’s realized that it would be better to have a mutually beneficial or at least tolerant relationship rather than one where they murder all of us?

Really? I've seen much more discourse around job displacement, "permanent underclass", loss of meaning, and cyber attacks, at least until recently with the HuggingFace stuff.

The problem is that all the former can still happen even if "the AIs decide to have a tolerant relationship rather than one where they murder all of us." It's all disruption caused by the technology moving way too fast for humans & society to adjust.

Re: I resigned from Anthropic today

#218

Post by foogazi »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

> I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity.

Nuclear weapons don’t have AI but AI can have nuclear weapons

Post reply on HN