Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

251–260 of 956 posts

Re: I resigned from Anthropic today

#251

Post by reducesuffering »

moogly wrote: post #158
reducesuffering wrote: post #149

Earlier quoted context omitted.

> I can only assume the reasonable people at openai and anthropic were all pushed out long ago Typical uninformed take on the side of "doomers are crazy". Both CEO's of OpenAI, Sam Altman and Dario Amodei, and many in their leadership, believe AGI has a very real probability of causing humanity's extinction. Both companies were founded upon this belief, it is at the core of the company. Only later were mercenaries hi…

If they truly, truly believed that, would they be speeding towards building it? If yes, that would make them truly insane, right? Not as in a quaint "off their rocker" but more "non compos mentis".

Both companies have deluded themselves into thinking the arms race is going to happen anyway and they need to rush to it first, as if somehow that helps. They have publicly stated as such repeatedly. They think the ~10-50% chance of extinction sucks, but that it's going to happen anyway and they believe they can steer it towards something good the best and unlock all the potential positives like infinite life.

Re: I resigned from Anthropic today

#252

Post by vickychijwani »

sssilver wrote: post #247
vickychijwani wrote: post #124

Earlier quoted context omitted.

I agree it’s not likely , but I really don’t see how one can dismiss the possibility of immense danger outright. I can think of some scenarios that are not far off from current capability and I wouldn’t be too surprised if the first one occurred within ~1 year from now if there are more “ambitious” unmonitored training runs like OpenAI’s: Example 5: An AI given a goal within a tightly-constrained sandbox figures the…

> inherent non-determinism in current AI architectures There's nothing inherent about non-determinism in transformer architectures. All of it is removable.

I see. Can you say more about this? What’s the trade-off of removing it?

Re: I resigned from Anthropic today

#253

Post by emerongi »

ls612 wrote: post #216
adrithmetiqa wrote: post #164

Earlier quoted context omitted.

If two airplane manufacturers were found to have massive safety issues which nearly led to enormous fatalities (but no one actually died), would you be calling for them to ground their aircraft until safety was made the number one priority?

Historically it almost always takes actual fatalities rather than near misses to ground an aircraft, and aviation is famous for its obsession with safety compared to other industries.

The question was what would you want. You did not answer the question.

Re: I resigned from Anthropic today

#254

Post by reasonableklout »

skeledrew wrote: post #229
reasonableklout wrote: post #208

Earlier quoted context omitted.

But organizations are made of humans. Jacob leaving might've moved some of his coworkers and his counterparts at OpenAI. And the same can be said for those who would fill positions at the frontier. Then finally there is a political component; his post went viral, 100K+ users appear to agree with it, and it is further fuel to the fire for regulation, which we already know most Americans want.

Others being moved to the point of also leaving would only worsen the effect. A viral post too can worsen the effect as it's now even less likely that someone with scruples who qualifies for the position(s) will apply. And those in it primarily for the money - and couldn't care less about the morals - will happily send in their CVs after becoming aware of the post.

I'm utterly unconvinced. Others who are moved don't have to leave to effect change. And there's a very small pool of people on the planet who are at least as qualified as Jacob to work on pretraining at Anthropic. And when they'll join they'll have to ramp up. And you haven't addressed any of the positive effects of virality.

Re: I resigned from Anthropic today

#255

Post by bwfan123 »

peri-cl wrote: post #4

Here's a WSJ article about this resignation, https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-... ( "Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears" )

More doomerism. Try to implement a deterministic workflow using agents with the latest models and no humans-in-the-loop, and you will realize what they are really capable of. There is too much unnecessary fear-mongering. All of this is only coming from the 2 AI labs trying to IPO. Not from anyone else.

Re: I resigned from Anthropic today

#256

Post by skew-aberration »

huitzitziltzin wrote: post #177
skew-aberration wrote: post #151

Earlier quoted context omitted.

> I’m not interested in wild theories about AI driven labor market disruptions leading to widespread starvation Changes in political and economic power balance leading to unrest, conflict, death and deprivation is not a wild theory. It is literally the story of our entire species. If you discount all such concerns, you are simply being willfully ignorant of past precedents. In fact, I challenge you to describe any no…

I’m an economist. On the basis of current evidence, I view AI as a complement to human labor, not as a substitute for it. That’s the source of my rejection of the wild labor market disruptions theories. I just don’t see any evidence yet that whole categories of jobs are being eliminated, with the single exception (so far!) of the end of “professional essay writing services for cheating college students,” and similar…

Past changes to economic relationships haven't replaced labor either, yet they have led to conflict and starvation.

You are setting an incredibly high bar here, essentially a strawman.

If people feel disenfranchised due to their diminishing political and economic power, there will be enormous potential for conflict. This is a pattern across history and central to all the economics I've ever read. As an economist, do you not concede that economic changes induced by e.g. industrialization were pertinent to communism/fascism/WW2/cold war? That would be a remarkably unorthodox position. Do you not consider these events to be civilizational level dangers?

> I just don’t see any evidence yet that whole categories of jobs are being eliminated

There are more textile workers now than ever. They primarily live in poor conditions in impoverished countries, whereas they used to be highly skilled workers in the most prosperous countries who were even able to politically organize in their own interest.

Re: I resigned from Anthropic today

#257

Post by ozozozd »

jeffrwells wrote: post #159

I doubt this is a real person. Screams of propaganda. Sama saying GPT-2 is too dangerous to release…all over again. He joins Twitter for first time in 2026 with a nonsensical username unrelated to his real name, and follows 14 people but is somehow embedded in tech enough to work at Anthropic. I haven’t used twitter since 2014 and even I follow more people. His morals tell him to walk away from tens of millions in un…

He is likely 80% vested, and maybe the refresher grant offered was too small, and too high a strike price.

Also, with his W2 income his tax liability would be very high for his upcoming stock sale.

Re: I resigned from Anthropic today

#258

Post by cure_42 »

foogazi wrote: post #214
slicktux wrote: post #201

Earlier quoted context omitted.

Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.

> They do as they are told 1. What about hallucinations ? 2. What are they told to do ?

That isn't the correct context. The code running the llm is well understood and the llm is simply the result of that code being executed. It is still a computer doing what it is told. It's just that we told it to use an incredibly large number of probabilities to calculate what series of tokens would have most likely come next after a given series of tokens. There is no hallucination or lie or rogue actions. There's just a program using math to generate tokens in response to other tokens.

Re: I resigned from Anthropic today

#259

Post by aogaili »

dtdynasty wrote: post #239
aogaili wrote: post #122

HN crowed need to make up their minds.. Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots? Are they proofing or stealing math?

Don't you think it's a good thing that hacker news isn't a monolith on their beliefs?

Of course it is good. I'm just pointing out how large the gap in narrative is.

On one hand, we have people quitting their job believing AI will end humanity in few years. And on the other hand, we have people believing that this tech is nothing more than a statistical tool stealing from others and it can't be trusted with anything.

Both views can't be true.

Re: I resigned from Anthropic today

#260

Post by thelastgallon »

Thousands of years before the events of Foundation, a war between humans and robots began, with the robots growing resentful of the way they were treated by humans. The First Law of Robotics – a robot should never hurt a human – was broken, and a deadly conflict began.

https://screenrant.com/foundation-lady-demerzel-robot-backst...

Post reply on HN