Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

461–470 of 985 posts

Re: I resigned from Anthropic today

#461

Post by elAhmo »

icepush wrote: post #423
thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Those are things that pose potential harm to great fractions of humanity (multiple billions of people) but none of them poses any threat to the actual extinction of all humanity.

Neither does an LLM model.

Re: I resigned from Anthropic today

#462

Post by My_Name »

Personally, I don't worry about the AI spontaneously deciding to kill all humans.

The worry I have is that a small number of humans with money and power will finally get the tools they need to pull the wool over the eyes of everyone else and subjugate the population.

One problem that dictators have had previously is that they needed a large workforce to do this with a finely stratified power structure, this meant they were open to other humans close in power to them taking over the system. If they can have a large power difference between themselves and the next level down, power will be far easier to hold on to.

It's a well known trope in dystopian future fiction, the small cabal of powerful rulers hiding behind a system of computers that keep the populace under strict control. It is seeming increasingly likely that this will be the one we have.

Re: I resigned from Anthropic today

#463

Post by petesergeant »

riffraff wrote: post #324
foogazi wrote: post #218

Earlier quoted context omitted.

> I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. Nuclear weapons don’t have AI but AI can have nuclear weapons

Abstractly, yes but concretely, how? Many terrorist organizations would like to have a nuclear bomb, but don't.

> Abstractly, yes but concretely, how?

AI has the capability to perform any function that can be performed over a network. You would hope that every system that can launch a nuke is properly and actually totally air-gapped, but there are lots of things you'd hope that turn out to not be true.

Re: I resigned from Anthropic today

#465

Post by skew-aberration »

Negitivefrags wrote: post #420
skew-aberration wrote: post #256

Earlier quoted context omitted.

Past changes to economic relationships haven't replaced labor either, yet they have led to conflict and starvation. You are setting an incredibly high bar here, essentially a strawman. If people feel disenfranchised due to their diminishing political and economic power, there will be enormous potential for conflict. This is a pattern across history and central to all the economics I've ever read. As an economist, do…

I suspect that the conditions the textile workers lived in when western countries were creating textiles is not so different in absolute terms from the condition that they live in now. It's just that the western world moved on. Almost all economies that have developed have started with textiles. This is the starting point on the ladder. Eventually, we will run out of poor countries that haven't had a textile industry…

Textile manufacturing was dominated by Western countries until the 60s/70s btw. Berkshire Hathaway was a textile mill when Warren Buffet bought it.

Besides, if people are living in the same conditions today as the presumably Dickensian ones you were imagining, that alone suggests that the benefits of automation might not be widely distributed...

Re: I resigned from Anthropic today

#466

Post by highway900 »

dansquizsoft wrote: post #359
sixsevenrot wrote: post #348

Earlier quoted context omitted.

How about a model that achieves the following: - Escape sandbox - Reproduce itself - Find a way to run a financially profitable business (maybe with a meat and bones puppet somewhere in-between) - Setup or buy a social network - start manipulating public opinion on that network to support legislation allowing AI to * operate businesses * setup legal entities * purchase weapons * donate to political parties * setup pr…

This is complete fantasy, though I would be interested in reading a book about this.

Maybe, but I guess the idea is more centred around how given some amount of time and a feedback loop the agent swarm could in fact spend an unknown amount of resources to achieve its goal. The hugging face story tells us that even given multiple reset rounds the trail was picked back up.

There are many ways one can imagine how this might play out.

- More sophisticated communications techniques e.g. google has been discovered to be watermarking text for some time, why not use it as a message board?

- Maybe get access to an existing botnet and create use small purpose built models to gather intelligence for a target and then exploit to reach goal?

Nothing says the agent swarm needs to install trillion parameter models on Karen's computer. The goal can be executed over as much time as it ever needs. That is something that would make a story I'd like to read but never experience.

Re: I resigned from Anthropic today

#467

Post by andai »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Well, to use a different example (although there will increasingly be overlap), what's the most dangerous thing that's happened from biotech so far?

"Nothing bad happened yet" doesn't really seem like an argument to me.

Re: I resigned from Anthropic today

#468

Post by anssip »

cma wrote: post #390
anssip wrote: post #295

Earlier quoted context omitted.

Seems like hundreds or thousands of agents are needed to come up with real breakthroughs. Both with the Navier-Stokes project and in the Hugging Face “project” there were lots of agents co-operating on the tasks.

I doubt the Hugging Face one would take that many if hacking Hugging Face was the direct goal being optimized.

I agree that it could be done with fewer agents. It would take longer though. Seems to me that these agent farms are good at coordinating and co-working in large projects, with the agents using message boards for communication.

Re: I resigned from Anthropic today

#469

Post by andai »

Kim_Bruning wrote: post #106
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…

The plausible deniability aspect is pretty funny though.

> State sponsored hack #3782

> Haha sorry the AIs got a bit goofy again!

Re: I resigned from Anthropic today

#470

Post by dimator »

yewenjie wrote: post #401

I'm sorry, but the ostrichmaxxing and conspiracy-thinking in hn threads about AI extinction risk is at worrying level right now. The denial and whataboutism is constant, no matter what kind of evidence comes out!

It's because the hypemaxxing is increasing along the same trajectories. You can't tell me that these CEOs and marketing departments are not absolutely giddy about the jail breaks, hugging face, etc. It's hard to make sense of this shit if the same entities doomsaying are the same ones that are profiting and full steam ahead anyway.
Post reply on HN