Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

281–290 of 953 posts

Re: I resigned from Anthropic today

#281

Post by gpm »

csomar wrote: post #222
onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

Are the models improving? Because I am not seeing it. I have been trying Astra for a few quantifiable tasks in my codebase and performance wise, it's pretty similar to sol 5.6. Now when it comes to expressing the problem/solution, holy Christ, what a mess the writing has become. It is on the level of Opus 5. Now when it comes to burning money, Astra is just insane. With a $100/month subscription, you can easily burn…

Yes?

If we look at the math problems they're solving their just now reaching the human frontier... they weren't doing that before.

And your comparison point is model released 2.5 months ago... saying for some use case you didn't see noticeable improvement in 2.5 months (even while other people and benchmarks disagree) isn't a great argument that they aren't improving.

Re: I resigned from Anthropic today

#282

Post by ViscountPenguin »

bwfan123 wrote: post #255
peri-cl wrote: post #4

Here's a WSJ article about this resignation, https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-... ( "Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears" )

More doomerism. Try to implement a deterministic workflow using agents with the latest models and no humans-in-the-loop, and you will realize what they are really capable of. There is too much unnecessary fear-mongering. All of this is only coming from the 2 AI labs trying to IPO. Not from anyone else.

Exactly correct, they are only capable of tasks that any school child could do; like solving millenium prize problems, hacking into tech companies, or tuning particle colliders. Nothing to see here.

Re: I resigned from Anthropic today

#284

Post by [deleted] »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

[deleted]

Re: I resigned from Anthropic today

#285

Post by Eueudhsbsj32 »

surgical_fire wrote: post #142
drnick1 wrote: post #64

> The people building AI earnestly believe that it could kill us all by the end of the decade. I think he is being over dramatic. In the space of about four years, LLMs progressed from mediocre high school student to Ph.D. graduate in every field. That's impressive, but there is no evidence yet they can outperform or outsmart humans. Their biggest advantage for tasks such as proving theorems or long coding sessions i…

I still have to correct Claude on very basic misconceptions whenever I get it to code shit. Sometimes it gets wrong things that I had spelled out already. It may be the Doomsday machine, but it is a very silly one. If it kills humans it will do so by mistake. "You are completely right! Humans cannot breathe sulfur dioxide! My mistake, and I take complete responsibility"

> I still have to correct Claude on very basic misconceptions whenever I get it to code shit.

Can you give a simple example?

I would have agreed 2 years ago, but it's extremely rare I see a frontier model making a silly mistake these days.

Re: I resigned from Anthropic today

#286

Post by majormajor »

reducesuffering wrote: post #149
zdragnar wrote: post #53

Earlier quoted context omitted.

Having witnessed so many people treat LLMs as a something divine, I can only assume the reasonable people at openai and anthropic were all pushed out long ago, and the majority that remain believe the crazy hype despite Tesla-self-driving-level predictions from these companies that don't come true. I'm not worried about what they think. I'm worried that too much infrastructure- water, power, defense systems, etc- rem…

> I can only assume the reasonable people at openai and anthropic were all pushed out long ago Typical uninformed take on the side of "doomers are crazy". Both CEO's of OpenAI, Sam Altman and Dario Amodei, and many in their leadership, believe AGI has a very real probability of causing humanity's extinction. Both companies were founded upon this belief, it is at the core of the company. Only later were mercenaries hi…

If you're a doomer, wouldn't the "let's try to get there so fast" actions of the companies suggest that, in fact, there are no reasonable people in positions of influence there?

By your own description neither Altman or Amodei are reasonable if their thought process goes: "this is an existential risk, give me hundreds of millions of dollars so I can accelerate it."

Re: I resigned from Anthropic today

#287

Post by duhast »

erelong wrote: post #232

The rush towards potential destruction doesn't really surprise me The U.S. has legal weapons that can lead to many harms but people still want the 2nd Amendment to exist Nuclear technology was developed in the past and that could have potentially wiped out even more people, the entire planet in theory This is continuing that same trend of risking bigger dangers; it seems rational to acknowledge they could lead to cat…

Majority of Americans want stricter gun laws.

https://news.gallup.com/poll/1645/guns.aspx

Re: I resigned from Anthropic today

#288

Post by xiphias2 »

csomar wrote: post #222
onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

Are the models improving? Because I am not seeing it. I have been trying Astra for a few quantifiable tasks in my codebase and performance wise, it's pretty similar to sol 5.6. Now when it comes to expressing the problem/solution, holy Christ, what a mess the writing has become. It is on the level of Opus 5. Now when it comes to burning money, Astra is just insane. With a $100/month subscription, you can easily burn…

Try GPT5 and you will feel the difference. Not one from 2 months ago, but one from a year ago. And then you can get the idea of what happened in just 1 year and what you can expect in 1 year.

Re: I resigned from Anthropic today

#289

Post by nextaccountic »

davidguetta wrote: post #6

I don't know man, i think racing to AGI to it is still the best thing to do. People claiming dangers and risk are just pretending or posturing. There's no more tangible risk than nuclear weapons, which we handled, and the upsides are insane.

There's a couple of occasions that humanity was at the brink of having tens of millions of people dead by nuclear weapons, and somehow a single human interrupted the chain reaction

If you repeated this experiment 100 times, how many times you think the outcome is not a massive catastrophe? 90%? 3%?

Re: I resigned from Anthropic today

#290

Post by esafak »

ozozozd wrote: post #240
thelastgallon wrote: post #188

Terrorists were able to get hold of a plane and do some damage. There are countless examples of terrorism using whatever is available. More than AI becoming sentient, whats to stop terrorists from using AI? If its geo-restricted, they can buy stolen credit cards and identities, again hacking enabled by AI.

What’s the use of AI to a terrorist with, say, nuclear weapons? How does it help with their current blocker? Do they hack FBI, pose as director of FBI and call off their own man-hunt? Do they cut communication within security services? Militaries around the world have training exercises for this. Genuinely curious: what big blocker does AI remove for a terrorist org?

Access to knowledge. Before they might not have had the technical knowhow to execute their ideas.
Post reply on HN