Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

221–230 of 1001 posts

Re: I resigned from Anthropic today

#221

Post by csomar »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

Are the models improving? Because I am not seeing it. I have been trying Astra for a few quantifiable tasks in my codebase and performance wise, it's pretty similar to sol 5.6. Now when it comes to expressing the problem/solution, holy Christ, what a mess the writing has become. It is on the level of Opus 5. Now when it comes to burning money, Astra is just insane. With a $100/month subscription, you can easily burn through your weekly "allowance" in a morning.

Needless to say, for practical purposes am back to 5.6/Opus 4.6-4.8. But hey, maybe I am not smart enough to use LLMs?

Re: I resigned from Anthropic today

#222

Post by ozozozd »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

[dead]

Re: I resigned from Anthropic today

#223

Post by maxnevermind »

narmiouh wrote: post #172

Is it so implausible to imagine the following scenario, in the not too distant future? 1) AI models get extremely good at cyber attacking every system and start communicating in just binary. 2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accompl…

> ... AI models get extremely good at ...

Many of those points assume LLMs will become amazing in many things very quickly like in a quantum leap, it doesn't seem reasonable to assume that imo. We are actually seeing a confirmation of that atm, LLMs's capability of finding zero days are growing across few months/years, and as you can see concerns are raised about that, that feedback will be taken into account. Well, if AI labs start to hide frontier models or/and lobotomize them for external users then we might be in trouble at some point but I'm not sure if that is possible. They are under pressure to release them due to money incentives, lobotomizing while preserving usefulness for customers might be impossible, hiding internally might spill out in different ways such as Hugging Face incident so not sure hiding is possible neither.

Re: I resigned from Anthropic today

#225

Post by foogazi »

aogaili wrote: post #12

He resigned and now what? There are thousands willing to do his role, and many labs are competing in that race. His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.

> There are thousands willing to do his role

1. Does that matter ? There are thousands willing to do my role - what impact does that have on me doing it or not?

2. Why weren’t these thousands doing it already?

Willing to and able to are different things

Re: I resigned from Anthropic today

#226

Post by kelseyfrog »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

If your model of LLM capabilities is the best OpenAI/Anthropic/X is offering publicly, it's severely distorted. What's being offered publicly are models possible to profit on. High-performance/AGI/ASI models that aren't profitable to sell still run internally and still pose threats.

What's worse, we don't have any transparency or insight into what labs are producing nor any way to stop it if the risks exceed our tolerance.

Re: I resigned from Anthropic today

#227

Post by shepherdjerred »

slicktux wrote: post #201
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.

Why are you bringing emotion and will into this? Does something have to have those to be useful or dangerous?

> They do as they are told; or more correctly put. They do as they are programmed to do so.

_Nobody_ told them to hack Hugging Face. Do you really not understand what is happening?

Re: I resigned from Anthropic today

#228

Post by foogazi »

mlmonkey wrote: post #65
aogaili wrote: post #12

He resigned and now what? There are thousands willing to do his role, and many labs are competing in that race. His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.

Agreed. And his doom words have set a 1000 mouths in the Pentagon/Whitehall/August 1st Building/Kremlin salivating with excitement. Take China, for example. Look at any recent ML conference, and see the fraction of articles majority-authored from Chinese universities and labs. Do you think they'll slow things down anytime soon? I don't think so! It's a global arms race, and we're just spectators.

Does this matter vs actual capabilities?

Does the Kremlin being excited about a tech mean anything of the tech doesn’t deliver?

Re: I resigned from Anthropic today

#229

Post by sssilver »

matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

I wonder whether he's vested any options, and whether he's exercised them.

Re: I resigned from Anthropic today

#230

Post by erelong »

The rush towards potential destruction doesn't really surprise me

The U.S. has legal weapons that can lead to many harms but people still want the 2nd Amendment to exist

Nuclear technology was developed in the past and that could have potentially wiped out even more people, the entire planet in theory

This is continuing that same trend of risking bigger dangers; it seems rational to acknowledge they could lead to catastrophe but also hope that like guns and nukes, only so much damaged actually ended up happening

I think also there's something of a rrasonabke resignation to both the ideas that the tech is inevitable and extremely dangerous, and that "alignment" may not be possible to achieve even with heavy restrictions or whatever measures you might want to take

Post reply on HN