Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

641–650 of 938 posts

Re: I resigned from Anthropic today

#641

Post by thomascountz »

lf88 wrote: post #533
thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

AI progress is linked to most of these dangers, actually: - it will likely cause massive unemployment, leading to rampant wealth inequality - we are now seeing some use of autonomous weapons in real conflicts - increasingly relying on LLMs is arguably a form of technology dependence (and cognitive dependence) - datacenters have a non negligible environmental impact

I agree these dangers are connected. The danger comes from human institutions and incentive structures which (having existed long before now) are being exploited and exacerbated by LLMs, making them harder to ignore.

Re: I resigned from Anthropic today

#642

Post by PowerElectronix »

I fail to see how a machine that can hack everything can't also patch everything and make the system unhackable.

A nuke, a virus, whatever... The knowledge is nonlonger the bottleneck, it's the tools and materials.

Also, I'm extremely skeptical about AI becoming even close to a child in intelligence.

Re: I resigned from Anthropic today

#643

Post by ben_w »

geraneum wrote: post #545
thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

The AI perhaps wouldn't be such an "imminent threat" if not for wealth inequality, consolidation of power, monopoly, and the opaqueness of it all.

I think that mainly changes the nature of the threat rather than anything else.

Even if we didn't already have any wealth inequality or consolidated power, they're already starting to become capable of creating those things for the first people to think to ask for it. Even in little ways, like telling you how to make a laser microphone and then doing voice-to-text and sentiment analysis on all the voices you hear.

Remember, we had social networks before Zuckerberg monetised it and put ads in between every third message you saw from your friends; and not every third that they wrote, every third that Zuckerberg's ad system deigned to show you to keep you hooked.

We had machine learning systems even back then. It was still AI, it just wasn't able to evaluate or respond to freeform text.

Re: I resigned from Anthropic today

#644

Post by karlosvomacka »

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

none of the things you mentioned is agentic and could posses abilities beyond human level intelligence.

Re: I resigned from Anthropic today

#645

Post by [deleted] »

copperx wrote: post #153
matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

[deleted]

Re: I resigned from Anthropic today

#646

Post by applicative »

Kim_Bruning wrote: post #106
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…

The hugging face incident had no effect on hugging face, whose service is replicated by countless sites. The descriptions of the phases of the interaction are thrilling in a way that inclines one to forget this

Re: I resigned from Anthropic today

#647

Post by reasonableklout »

CodeCompost wrote: post #312

And the hype machine continues. I willing to bet that Anthropic asked him to make that post.

I don't understand how you can make this claim while working in software engineering and seeing how our field has utterly transformed over the last year, then the unrelenting march of astonishing breakthroughs and incidents this summer. It's like we're living on different planets. What would it take to convince you that the technology poses real, societal-scale risks and that people working at the labs genuinely believe what they say when they talk about them?

Re: I resigned from Anthropic today

#648

Post by vlyan »

par1970 wrote: post #476
vlyan wrote: post #416

Earlier quoted context omitted.

>After the events of the summer After the blatant marketing campaigns of the summer, you mean. do you need a reminder that those very same people had touted GPT-2 as a dangerous model? worrying about sci-fi doomsday scenarios with the current AI tech is absurd. LLMs predict the next token, that's literally all they do. they aren't going to escape into the cyberspace, self-replicate, self-improve, jump over air gaps a…

> those very same people had touted GPT-2 as a dangerous model Where did they say this at? AFAIK this is the original GPT-2 announcement: https://openai.com/index/better-language-models/ . Here are some direct quotes: “We can also imagine the application of these models for malicious purposes , including the following (or other applications we can’t yet anticipate): * Generate misleading news articles * Impersonate o…

yes, exactly, that's what they claimed that incoherent gibberish generator to be capable of.

Re: I resigned from Anthropic today

#649

Post by King-Aaron »

Dylan16807 wrote: post #596
King-Aaron wrote: post #564

Earlier quoted context omitted.

The thing that would end our species due to nuclear war is the exact same thing that would end us from climate change - a collapse of the modern systems of society that we depend on for survival. A nuclear exchange won't set every human on fire, but it's the collapse of food production, healthcare, logistics chains, security, and eventual disease that follows. And because of the size of our population and how depende…

Outside of extreme feedback loop scenarios, there is no way for climate change to disrupt food anywhere near the level of nuclear winter, and there would be no mass destruction of supply chains either.

Welp extreme feedback loops are indeed on the menu.

Re: I resigned from Anthropic today

#650

Post by xienze »

thomascountz wrote: post #456
icepush wrote: post #423

Earlier quoted context omitted.

Those are things that pose potential harm to great fractions of humanity (multiple billions of people) but none of them poses any threat to the actual extinction of all humanity.

This is not correct. Please go through each one again, taking special note of global warming and nuclear weapon development .

Humans are voluntarily driving themselves to extinction through sub-replacement birth rates worldwide. That's going to play out far quicker than global warming or pretty much anything other than every country on earth launching nukes at each other will.
Post reply on HN