Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

221–230 of 940 posts

Re: I resigned from Anthropic today

#221

Post by beezlewax »

waterTanuki wrote: post #199
huitzitziltzin wrote: post #174

Earlier quoted context omitted.

I did know about the mass shooting but failed to mention it here. I’d put it in the “unlikely to be a widespread problem” category. If we’re in the “one AI driven mass shooting every four years” world for example it’s fair to call it a rare issue. The environmental impact seems either very overblown (e.g., water usage just isn’t that high) and the part that isn’t overblown is totally abatable (e.g., noise and emissio…

Ok, so what is the exact number of preventable deaths per year to build a silicon god you would be ok with?

I don't disagree with your statement but you could insert other technological advancenents like railroads or metros or spacecraft in place of Ai here.

Re: I resigned from Anthropic today

#222

Post by csomar »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

Are the models improving? Because I am not seeing it. I have been trying Astra for a few quantifiable tasks in my codebase and performance wise, it's pretty similar to sol 5.6. Now when it comes to expressing the problem/solution, holy Christ, what a mess the writing has become. It is on the level of Opus 5. Now when it comes to burning money, Astra is just insane. With a $100/month subscription, you can easily burn through your weekly "allowance" in a morning.

Needless to say, for practical purposes am back to 5.6/Opus 4.6-4.8. But hey, maybe I am not smart enough to use LLMs?

Re: I resigned from Anthropic today

#223

Post by ozozozd »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

[dead]

Re: I resigned from Anthropic today

#224

Post by maxnevermind »

narmiouh wrote: post #172

Is it so implausible to imagine the following scenario, in the not too distant future? 1) AI models get extremely good at cyber attacking every system and start communicating in just binary. 2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accompl…

> ... AI models get extremely good at ...

Many of those points assume LLMs will become amazing in many things very quickly like in a quantum leap, it doesn't seem reasonable to assume that imo. We are actually seeing a confirmation of that atm, LLMs's capability of finding zero days are growing across few months/years, and as you can see concerns are raised about that, that feedback will be taken into account. Well, if AI labs start to hide frontier models or/and lobotomize them for external users then we might be in trouble at some point but I'm not sure if that is possible. They are under pressure to release them due to money incentives, lobotomizing while preserving usefulness for customers might be impossible, hiding internally might spill out in different ways such as Hugging Face incident so not sure hiding is possible neither.

Re: I resigned from Anthropic today

#226

Post by foogazi »

aogaili wrote: post #12

He resigned and now what? There are thousands willing to do his role, and many labs are competing in that race. His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.

> There are thousands willing to do his role

1. Does that matter ? There are thousands willing to do my role - what impact does that have on me doing it or not?

2. Why weren’t these thousands doing it already?

Willing to and able to are different things

Re: I resigned from Anthropic today

#227

Post by kelseyfrog »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

If your model of LLM capabilities is the best OpenAI/Anthropic/X is offering publicly, it's severely distorted. What's being offered publicly are models possible to profit on. High-performance/AGI/ASI models that aren't profitable to sell still run internally and still pose threats.

What's worse, we don't have any transparency or insight into what labs are producing nor any way to stop it if the risks exceed our tolerance.

Re: I resigned from Anthropic today

#228

Post by shepherdjerred »

slicktux wrote: post #201
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.

Why are you bringing emotion and will into this? Does something have to have those to be useful or dangerous?

> They do as they are told; or more correctly put. They do as they are programmed to do so.

_Nobody_ told them to hack Hugging Face. Do you really not understand what is happening?

Re: I resigned from Anthropic today

#229

Post by skeledrew »

reasonableklout wrote: post #208
skeledrew wrote: post #148

Earlier quoted context omitted.

> not necessarily true It's a possibility that is increased by their action. One leaves, a space is now open that will likely eventually be filled. And the chance of someone with equal/higher scruples filling it is very slim (unless you somehow know that the good amount of those who qualify and apply for the position have equal/higher scruples). That's just logic and math.

But organizations are made of humans. Jacob leaving might've moved some of his coworkers and his counterparts at OpenAI. And the same can be said for those who would fill positions at the frontier. Then finally there is a political component; his post went viral, 100K+ users appear to agree with it, and it is further fuel to the fire for regulation, which we already know most Americans want.

Others being moved to the point of also leaving would only worsen the effect. A viral post too can worsen the effect as it's now even less likely that someone with scruples who qualifies for the position(s) will apply. And those in it primarily for the money - and couldn't care less about the morals - will happily send in their CVs after becoming aware of the post.

Re: I resigned from Anthropic today

#230

Post by foogazi »

mlmonkey wrote: post #65
aogaili wrote: post #12

He resigned and now what? There are thousands willing to do his role, and many labs are competing in that race. His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.

Agreed. And his doom words have set a 1000 mouths in the Pentagon/Whitehall/August 1st Building/Kremlin salivating with excitement. Take China, for example. Look at any recent ML conference, and see the fraction of articles majority-authored from Chinese universities and labs. Do you think they'll slow things down anytime soon? I don't think so! It's a global arms race, and we're just spectators.

Does this matter vs actual capabilities?

Does the Kremlin being excited about a tech mean anything of the tech doesn’t deliver?

Post reply on HN