Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

791–800 of 1001 posts

Re: I resigned from Anthropic today

#791

Post by henryaj »

frabcus wrote: post #641
henryaj wrote: post #406

Earlier quoted context omitted.

Hugging Face incident, Anthropic reporting sandbox escape, AISI reporting models trying to push exploits to the wild

Also (and under-reported, so you could easily have missed it) OpenAI's agents got access to K8 admin on their own research cluster. "This escalation also yielded access to OpenAI’s managed cloud Kubernetes service. The agents escalated to Kubernetes cluster-admin and created a privileged host-mounted pod" https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c78... (see section V)

Just baffles me when people are like "well no-one's died yet". How long until some mission-critical system is compromised, or hackers use LLMs to ransom a hospital chain?

Re: I resigned from Anthropic today

#792

Post by vyyy »

epihelix wrote: post #301
platinumrad wrote: post #97

Earlier quoted context omitted.

Anthropic is a company full of basilisk believers.

Yes, but the really weird thing is that they seem to: a) believe that what they're creating is a basilisk, and b) keep trying harder to do this while staring right at it I think they're very deluded about (a) -- but if they do actually believe this (and it really seems like a decent proportion of Anthropic truly does), then why keep doing (b)? That seems to be why this individual resigned, but I'm surprised it's not…

[flagged]

Re: I resigned from Anthropic today

#793

Post by vyyy »

danaris wrote: post #506
epihelix wrote: post #301

Earlier quoted context omitted.

Yes, but the really weird thing is that they seem to: a) believe that what they're creating is a basilisk, and b) keep trying harder to do this while staring right at it I think they're very deluded about (a) -- but if they do actually believe this (and it really seems like a decent proportion of Anthropic truly does), then why keep doing (b)? That seems to be why this individual resigned, but I'm surprised it's not…

With Roko's Basilisk, if you believe in it, the most rational thing to do is to put forth every effort to bring it into being. Because if you don't, then you will be one of its targets when it does, inevitably, come into being. (I am not a basilisk believer. I think this is all absolute horseshit. But to understand someone's motivations, one must think like them.)

[flagged]

Re: I resigned from Anthropic today

#794

Post by benibela »

dansquizsoft wrote: post #355
sixsevenrot wrote: post #344

Earlier quoted context omitted.

How about a model that achieves the following: - Escape sandbox - Reproduce itself - Find a way to run a financially profitable business (maybe with a meat and bones puppet somewhere in-between) - Setup or buy a social network - start manipulating public opinion on that network to support legislation allowing AI to * operate businesses * setup legal entities * purchase weapons * donate to political parties * setup pr…

This is complete fantasy, though I would be interested in reading a book about this.

>a book about this.

Daemon+Freedom by Daniel Suarez

though it is kind of mid

Re: I resigned from Anthropic today

#795

Post by scott_weber »

ben_w wrote: post #645
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

> I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. Even today, AI is the biggest discrete threat to bringing carbon emissions under control. Almost everyone wants to decarbonise… except Trump. Renewables are the cheapest source of new power… but the demand for new electricity for the data centres is so high that all options are on t…

The classic worry is https://en.wikipedia.org/wiki/Nuclear_winter: the soot thrown up by the ensuing urban fire storms causing temperatures, rainfall to plummet, and hence food production. On reading the scenarios the odds of total human extinction are lower than I recollected, but 80% of world pop dying of starvation dwarfs the direct death toll (usually reckoned in the hundreds of millions)

Re: I resigned from Anthropic today

#796

Post by ungovernableCat »

frabcus wrote: post #661
gizmodo59 wrote: post #495

Earlier quoted context omitted.

This is a lame argument. Questioning his financial incentive is very legit. Why should we trust him?

Questioning his finances is a poor argument because he clearly would have more money if he had stayed at Anthropic for the next couple of years, than he will have by leaving. Even if he still has vested options, or savings from his salary or whatever, those would be larger if he stayed.

I think it’s a valid question to ask if he quit the company for moral reasons.

If you seriously think the frontier labs are dangerously playing with everyone’s lives (how many companies can even claim this civilisational scale) then why would you be ok with holding vested shares which will grow in value if the corporation achieves what it aims to achieve?

At the very least you can sell them (they’re not so illiquid considering the company’s hype and success) and invest in the broad market.

Re: I resigned from Anthropic today

#798

Post by nullbio »

wallst07 wrote: post #784
nullbio wrote: post #539

Earlier quoted context omitted.

Literally every single thing they do and say leads me to believe the scenario I described would be a fantasy for them. They're a radical cult collectively blinded by delusions of grandeur and a moral superiority complex who genuinely believe they are the only ones capable of wielding the proverbial sword.

How so?

It's a long story. If you're immersed in this world and aware of the movements all of the major AI players (in particular, Anthropic) have been doing over the last 3 years it paints a very clear picture. The patterns of behavior (the lying, cheating, stealing, the gaslighting, the virtue signalling, the grandstanding) speak for themselves.

Re: I resigned from Anthropic today

#799

Post by scott_weber »

TrackerFF wrote: post #746
thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Let's just for the sake of discussion assume that one time in the future, near or distant, AI manages to become sentient. And like other forms of life, its main motivation is survival: Like biological life competes for food and land, AI competes for power and compute. Probably the first motivation would be to find ways not to lose control over itself (i.e. remove human ability to control it), find ways not to lose en…

> AI decides that energy spent toward human agriculture is less important than work spent toward storage and production of energy

I mean. Don't need rogue AI for this. These are already the policies of those in charge of it: human needs are secondary to power and profit.

So you know what. Maybe a rogue AI will be an improvement.

Re: I resigned from Anthropic today

#800

Post by rurban »

kakacik wrote: post #707
rurban wrote: post #604

Earlier quoted context omitted.

Didn't you follow the robotics advances in the Ukrainian - Russian battlefield? There is now a death zone of 50km, only controlled by drones and automatic weapons.

> There is now a death zone of 50km, only controlled by drones and automatic weapons. If there would be such 50km death zone, front line would not move a nanometer in a year, would it. You yourself contradict above in your next post. No need for being too dramatic, facts are enough here. In reality, frontline is moving constantly albeit by small chunks, russians are advancing a bit, getting beating elsewhere and so o…

https://sound.orf.at/podcast/oe1/oe1-journale---gehoert-vert... [german]

I'll take the expert Oberst Markus Reisner from the Austrian Army over anyone else.

Post reply on HN