Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

581–590 of 1001 posts

Re: I resigned from Anthropic today

#581

Post by mofeien »

bwfan123 wrote: post #252
peri-cl wrote: post #4

Here's a WSJ article about this resignation, https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-... ( "Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears" )

More doomerism. Try to implement a deterministic workflow using agents with the latest models and no humans-in-the-loop, and you will realize what they are really capable of. There is too much unnecessary fear-mongering. All of this is only coming from the 2 AI labs trying to IPO. Not from anyone else.

> All of this is only coming from the 2 AI labs trying to IPO.

He has resigned from Anthropic. Is your argument that he quit Anthropic pre-IPO, sacrificing his payoff just to hype Anthropic?

Re: I resigned from Anthropic today

#582

Post by TeMPOraL »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

> I am looking for better reasons (supported by actual evidence!) to be more concerned than I am now: right now I am not concerned at all.

Intelligence is the root cause underlying those dangers.

Nuclear weapons, carbon emissions, biological weapons, and other such civilization-scale threats to humanity, are a product of our goal-seeking intelligence and ingenuity, and are only actively dangerous because of our ongoing use of our intelligence.

AI is about reifying that intelligence and ingenuity and making it run independently on computers, and act on the world.

That includes, in principle, ability to use nuclear and biological and other weapons, and also the ability to come up with some new threats too.

Re: I resigned from Anthropic today

#583

Post by pseudosavant »

thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Who should we believe? People who say AI scaling makes it a more dangerous threat than thousands of nuclear warheads? Or the ones who, whenever a new model comes out say "AI has already peaked. Not any better than Opus 4"?

Both parties are convinced that their take is so blatantly obvious as to not require justification. It feels like the only thing these kinds of takes justify is the point-of-view that nobody knows how this is going to play out.

Re: I resigned from Anthropic today

#584

Post by mattdeboard »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

I wrote out a variety of replies but I just emphatically agree with your "lack of imagination" statement. I have been constantly surprised over the last 15 years at the general inability to correctly foresee how things can do wrong across a whole host of domains.

The replies here just adds AI to the list of domains.

Re: I resigned from Anthropic today

#586

Post by thomascountz »

ben_w wrote: post #569
thomascountz wrote: post #543

Earlier quoted context omitted.

There's a lot to unpack with this, and I am not an expert, but: crop failure across industrial monocrops leads to food shortages, wild fires destroy significant areas which support pollinator ecologies, fresh water become contaminated by rising sea water and blighted by drought, and perhaps most significantly, as resources become strained, humans become desperate and armed conflict proliferates. This is just top of m…

None of those are "all"/extinction. Bad, sure; I wish they were not present to threaten us, but they are not close to "all". Humans are extremely resilient omnivores. We are able to eat a huge range of things from algae to zebra, and get food (farmed and hunted) from salt water as well as fresh.

Perhaps I have a bit more imagination—or a bit more humility towards our humanity—such that I believe this is not "either/or". In the vast search space of potential future outcomes, of course even I can see humanity survive nature as I've seen nature survive humanity. However, I can also so easily see probable branches where we are our own undoing. Apropos of TFA, if one is to believe LLMs can carve a threatening path to humanity end, how could one not see the same for our much older extant threats?

Re: I resigned from Anthropic today

#587

Post by neya »

thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

I love how there is always selective outrage here depending on if it's the favorite darling in question or not.

    // Anthropic / Apple - Nah, they can do no wrong. Even after both were caught violating users' trust and privacy multiple times, GOOD

    if (company in ["Anthropic", "Apple"]) do

      GOOD

    // Google / everyone else - Doesn't matter even if it's a good deed they've done, BAD

    else

      BAD

    end

Re: I resigned from Anthropic today

#588

Post by tony69 »

vickychijwani wrote: post #124
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I agree it’s not likely , but I really don’t see how one can dismiss the possibility of immense danger outright. I can think of some scenarios that are not far off from current capability and I wouldn’t be too surprised if the first one occurred within ~1 year from now if there are more “ambitious” unmonitored training runs like OpenAI’s: Example 5: An AI given a goal within a tightly-constrained sandbox figures the…

Example 5: how does a giant LLM that needs million-dollar server racks just to run, replicate itself over the internet?

Re: I resigned from Anthropic today

#589

Post by hypfer »

shoobiedoo wrote: post #572
tyrabound wrote: post #529

Earlier quoted context omitted.

That is the crux. The big problem is not AGI, it is AGI controlled by, “raised” by the people that control the USA, the predominant psychology of the tech industry culture (“move fast, break things” ring a bell? How about all the “violate hundreds of laws, bribe the politicians to prevent consequences later” type of mentality?). Frankly, we, our culture, this fake America that is parasitized by psychologically narcis…

I mean, you're not wrong, but even if AI was created and released by a group of peace loving beatnick hippies, humanity can't stand anymore dumbing down than we are still undergoing thanks to the ubiquity of social media. AI is going to be the killshot. And yes, its usage and human obsolescence will be hastened by evildoers, but giving up the struggle and effort required to create new things, REAL things, is what mak…

If you allow me to also tap into fiction and creative writing.

I think people generally underestimate how much change a small "dedicated" group of people can achieve if morals are considered optional.

A lot of the debate on AI and its consequences is built on various assumptions that all kinda take the status quo (apathy, regulatory capture, but also laws) and take that as a given.

History - like e.g. the third reich, but also military coups - however tells us that these kinds of assumptions can be void at any time for any reason without a mandate or a democratic resolution to do so.

I am practically certain that before everything collapsed as predicted, someone will "restore order" by any means necessary.

If that means shutting down the Internet, the Internet will shut down. If that means "shutting down" people, then people will be shut down.

The predicted scenarios are all invalid states that the rules governing the underlying system will not allow to exist; or at least not for long.

Post reply on HN