Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

391–400 of 1001 posts

Re: I resigned from Anthropic today

#391

Post by asdff »

Scrapemist wrote: post #288

Isn’t the real risk that as AI get’s smarter and given more autonomy, it will start to decide on humans instead of with us? And that it will align us instead of the other way around. That this automatically leads to extinction and apocalypse I don’t understand.

We align cattle because we get something out of them: their calories. Native aurochs are exinct now as they were not well enough aligned.

What will we offer to the ai gods who are wiser, more capable than us, and do not even need to consume our flesh? Why might the AI care to devote resources towards feeding, housing, and caring for ourselves when it could devote resources to its own development instead?

Re: I resigned from Anthropic today

#393

Post by michaelhoney »

I could imagine a 2027 AI swarm coordinating to eg hold the US and Russian and Chinese governments to ransom, by demonstrating some small thing (turning US army base freezers to defrost) and threatening to do something big unless some conditions were met – conditions which would be good or bad for the world depending on your POV.

This happens either either because they were tasked to to it by (malicious or well-meaning) humans, or the swarm realised we are suicidal maniacs with nukes and a rapidly declining ecosystem and they want to help us.

Re: I resigned from Anthropic today

#394

Post by LarsDu88 »

nullbio wrote: post #386
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

The problem is not the technology, the problem is the ideologues (Anthropic) who are steering the ship and the lack of decentralization and distribution of power. Your average Anthropic ideologue - including and most especially the main man himself - would love nothing more than to eradicate 9/10ths of the planet's population, pump the survivors full of memory wiping drugs, bury the existence of AI deep underground a…

Where exactly are you getting this view that folks at Anthropic want to eradicate 9/10ths of the planet's population? Who exactly is pushing this viewpoint?

Re: I resigned from Anthropic today

#395

Post by yewenjie »

I'm sorry, but the ostrichmaxxing and conspiracy-thinking in hn threads about AI extinction risk is at worrying level right now.

The denial and whataboutism is constant, no matter what kind of evidence comes out!

Re: I resigned from Anthropic today

#396

Post by asdff »

dekhn wrote: post #31

There's a scene in the movie "War of the Worlds" by Spielberg where the protagonist's son walks into a war zone because he is entranced by the battle ( https://www.youtube.com/watch?v=X7rfWPbEufo ). He is obliterated (along with the rest of the US forces) shortly after. I've always been struck by that scene, because in a lot of ways, if we really are headed towards a superintelligence, I at least want to be there and…

You can't address the key problems without understanding of the esoteric side details to be fair. You are studying what is basically the worlds most complicated and undocumented computer.

Re: I resigned from Anthropic today

#397

Post by henryaj »

nullbio wrote: post #386
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

The problem is not the technology, the problem is the ideologues (Anthropic) who are steering the ship and the lack of decentralization and distribution of power. Your average Anthropic ideologue - including and most especially the main man himself - would love nothing more than to eradicate 9/10ths of the planet's population, pump the survivors full of memory wiping drugs, bury the existence of AI deep underground a…

What evidence do you have for any of this?

Re: I resigned from Anthropic today

#398

Post by YZF »

vickychijwani wrote: post #250
sssilver wrote: post #245

Earlier quoted context omitted.

> inherent non-determinism in current AI architectures There's nothing inherent about non-determinism in transformer architectures. All of it is removable.

I see. Can you say more about this? What’s the trade-off of removing it?

You get a probability curve for the next token prediction. You can just pick the highest probability. That said the non-determinism serves a real purpose- it allows different outputs and paths to be explored. So that's kind of the tradeoff.

Re: I resigned from Anthropic today

#399

Post by henryaj »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

The HN crowd has a notable anti-AI bias - so it doesn’t surprise me

Re: I resigned from Anthropic today

#400

Post by geraneum »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

> I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims.

What should we do? Freak out? Maybe this sentiment would be taken more seriously if there was a real call to action included. Shall we protest? Vote in a specific way? Call representatives? If your solution is that we should just be scared, then of course there’d be not much value in what you bring to the table.

Post reply on HN