Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

471–480 of 1001 posts

Re: I resigned from Anthropic today

#471

Post by AIorNot »

platinumrad wrote: post #97
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Anthropic is a company full of basilisk believers.

You mean a effing cult like heavens gate.. call all this rationalist crap for what it is - a religous movement with leaders and prophets and even a demiurge like God

Re: I resigned from Anthropic today

#472

Post by hypfer »

andai wrote: post #419

> A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk. Which means they have to go faster, which means less responsibly? I he…

I believe that the actual alignment happens in.. uh.. "meatspace".

Someone is prompting. Someone is hosting.

That someone needs to be accountable for what happens. That someone needs to bleed if stuff goes haywire.

Humans at large have been "aligned" by the shared fear of death, pain and suffering. This has proven to work for millennia, so all we need to do is reapply it.

Re: I resigned from Anthropic today

#473

Post by andai »

chewbacha wrote: post #80

The most optimistic outcome of generative AI leaves us with a technology that warps our perception of reality and crushes labor. The most pessimistic destroys all of humanity. Our CEOs not only insist we genuflect before these machines but measure our sacrifice and shame our reluctance.

The children yearn for the mines.

Re: I resigned from Anthropic today

#474

Post by Synthetic7346 »

BLKNSLVR wrote: post #324
drivebyhooting wrote: post #52

Earlier quoted context omitted.

I was about to say something similar. If my cushy ad tech disappears (as it seems to be doing), I might as well join in with bringing about the end of all professions.

> ad tech disappears I pray for the day.

Can't come soon enough

Re: I resigned from Anthropic today

#475

Post by entropyneur »

asdff wrote: post #391
Scrapemist wrote: post #288

Isn’t the real risk that as AI get’s smarter and given more autonomy, it will start to decide on humans instead of with us? And that it will align us instead of the other way around. That this automatically leads to extinction and apocalypse I don’t understand.

We align cattle because we get something out of them: their calories. Native aurochs are exinct now as they were not well enough aligned. What will we offer to the ai gods who are wiser, more capable than us, and do not even need to consume our flesh? Why might the AI care to devote resources towards feeding, housing, and caring for ourselves when it could devote resources to its own development instead?

Also, extinction is WAY nicer than the cattle treatment.

Re: I resigned from Anthropic today

#476

Post by isolatedsystem »

emtel wrote: post #432
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Do you think people should continue developing AI up until the point that there is evidence that AI is facilitating biological weapons development? I think we should stop before then. But that necessarily means that there will not be evidence at the time that we stop.

This is rubbish. By that token, computer development is also facilitating biological weapons development. A better MacOS (or Windows, I don't know) leads to better weapons. They should clearly stop developing computers and OSes. Developers of nice test-tubes are also facilitating bioweapons. Your local O-ring manufacturer, your local medical-grade freezer manufacturer etc. are all culpable. The problem is the bioweapon, not the LLM.

Re: I resigned from Anthropic today

#477

Post by cybrox »

parl_match wrote: post #465
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

> Non-example 2: There are worries about AI-facilitated biological weapons. I haven’t seen any evidence that’s happening. https://www.science.org/content/article/made-order-bioweapon... Being able to use AI to generate the steps to synthesize proteins means that you can use it to use it to generate the steps to synthesize known toxins. Suddenly, once difficult to attain knowledge is now available to everyone.

Levelling the playing field either backfires spectacularly or increases overall safety dramatically.

Re: I resigned from Anthropic today

#478

Post by spockz »

xvedejas wrote: post #373
dansquizsoft wrote: post #355

Earlier quoted context omitted.

This is complete fantasy, though I would be interested in reading a book about this.

It has been interesting to me how AI has for many years given people a way to justify any worst case scenario. Nothing is too far-fetched if at any step you tell yourself the AI will be smarter than you, and thus be able to solve any conceivable obstacle. Oh, if only intelligence were the only bottleneck to power.

It is actually: knowledge, intelligence, communication, and a form of secrecy.

A single intelligent person is bounded in its reach by the network of people they can form around them and by which important/powerful people they can influence either directly or indirectly.

The idea behind the AI breakout is that it’s intelligence is maybe limited, but it will be able to quickly spread and covertly bring many devices into its influence. We already have the tools for formally letting agents talk to each other so it can also create a topology of agents that consists of cells where need to know is applied etc.

There are many forms of power some more obvious, say nation leaders, versus more silent powers such as career politicians or wealthy families or people controlling the media and thence the flow of information.

Re: I resigned from Anthropic today

#479

Post by andai »

hypfer wrote: post #472
andai wrote: post #419

> A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk. Which means they have to go faster, which means less responsibly? I he…

I believe that the actual alignment happens in.. uh.. "meatspace". Someone is prompting. Someone is hosting. That someone needs to be accountable for what happens. That someone needs to bleed if stuff goes haywire. Humans at large have been "aligned" by the shared fear of death, pain and suffering. This has proven to work for millennia, so all we need to do is reapply it.

You're still thinking in terms of control. That's the wrong model here. How do you control someone infinitely smarter than you? How do you control ten trillion someones?

Re: I resigned from Anthropic today

#480

Post by Ekaros »

parl_match wrote: post #465
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

> Non-example 2: There are worries about AI-facilitated biological weapons. I haven’t seen any evidence that’s happening. https://www.science.org/content/article/made-order-bioweapon... Being able to use AI to generate the steps to synthesize proteins means that you can use it to use it to generate the steps to synthesize known toxins. Suddenly, once difficult to attain knowledge is now available to everyone.

Still need to do it after getting those instructions. Not to forget equipment and precursors. I think just getting list of steps won't make it too much easier. Getting it mostly right is quite hard in many cases. And then with trivial cases you wouldn't even need AI. But just find something already documented.
Post reply on HN