Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

711–720 of 1001 posts

Re: I resigned from Anthropic today

#711

Post by darnfish »

PowerElectronix wrote: post #642

I fail to see how a machine that can hack everything can't also patch everything and make the system unhackable. A nuke, a virus, whatever... The knowledge is nonlonger the bottleneck, it's the tools and materials. Also, I'm extremely skeptical about AI becoming even close to a child in intelligence.

If you are attacking a system, you can try 100 things. If one thing works in getting access, you succeed.

If you are defending a system, you must defend against all 100 things. If one thing makes it through, you lose.

Re: I resigned from Anthropic today

#712

Post by kakacik »

xienze wrote: post #650
thomascountz wrote: post #456

Earlier quoted context omitted.

This is not correct. Please go through each one again, taking special note of global warming and nuclear weapon development .

Humans are voluntarily driving themselves to extinction through sub-replacement birth rates worldwide. That's going to play out far quicker than global warming or pretty much anything other than every country on earth launching nukes at each other will.

Hard disagree here, population growth is still happening and will keep happening for some time, maybe step out of your bubble if you don't see that.

Africa is still growing massively for example, world is not just western civilization. Sure at that rate and incerase of living conditions for everybody maybe in 1000 years population will be smaller, but its not that hard to fix if wanted - people used to have 10-15 kids as default.

Re: I resigned from Anthropic today

#713

Post by nairboon »

fergie wrote: post #704
matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

It should be obvious that AI is already capable of inducing humans to think or do things, and that this power is only going to increase over time.

It's not that much different from the effect of social media or the power held by other tech giants, who can control your exposure to particular information (media, search engines).

Re: I resigned from Anthropic today

#714

Post by frotaur »

jdthedisciple wrote: post #550
michaelhoney wrote: post #399

I could imagine a 2027 AI swarm coordinating to eg hold the US and Russian and Chinese governments to ransom, by demonstrating some small thing (turning US army base freezers to defrost) and threatening to do something big unless some conditions were met – conditions which would be good or bad for the world depending on your POV. This happens either either because they were tasked to to it by (malicious or well-meani…

And then we pull the plug, after holding our breath for 10 seconds,. Then life resumes normally..

In most AI takeover scenarios, if you take as a premise that the AI has human or above-human intelligence, and that it is misaligned, it is obviously aware of the pull the plug possibility.

Therefore, as you would if you were in its position, it will plan around it. For instance, by acting perfectly aligned for 2/3 years, continuing the improvement of its capabilities while being deployed in ever more systems.

Once it's confident it can act with high probability of success, it would then turn on us. This phenomenon is called 'treacherous turns'.

Any scenario in which you assume you have ASI or AGI but also find a 2-sentence way to foil the AI's plan is inconsistent, as the AI will also have thought of this failure mode.

Re: I resigned from Anthropic today

#716

Post by skinfaxi »

fny wrote: post #508

Humor me and suspend disbelief. If these models are such an existential threat to humanity, why are they controlled by two private companies? We might as well give Anthropic our nukes too.

> If these models are such an existential threat to humanity, why are they controlled by two private companies?

The government routinely contracts with private companies to create arms and munitions. Or do you think the bombs are delivered without payloads?

Re: I resigned from Anthropic today

#717

Post by dannersy »

eloisant wrote: post #706
Perseids wrote: post #655

Earlier quoted context omitted.

Let me argue on a technicality first: None of these are extinction level events. If global warming disrupts 99% of all crop production, the remaining 1% is still plenty enough to sustain a stable, if miserable, population. In fact you just need about 5k people for a stable gene pool[1]. Of the classical threats, only bioweapons got a shot at extinction, but even that is hard, given the (few) remaining truly secluded…

I still have to read a compelling argument on how AI will "extinct" humanity.

Indirectly as we offload our brains to the machine and we end up worshipping it because those who cared to understand it or be responsible were buried by capitalism of ages passed.

Re: I resigned from Anthropic today

#718

Post by rossant »

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

An AI, either acting autonomously or under human direction, hacks Russian/North Korea/etc. intelligence systems and convinces them that the US has launched ballistic missiles at them. The end.

Re: I resigned from Anthropic today

#719

Post by hakimg »

geraneum wrote: post #406
onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

> I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. What should we do? Freak out? Maybe this sentiment would be taken more seriously if there was a real call to action included. Shall we protest? Vote in a specific way? Call representatives? If your solution is that we should just be scared, then of course there’d be not much value in what you bring to the table.

Why does everything have to be so black and white? It always either there's no threat at all or we all need to panic. How about be open to a reasonable discussion on potential outcomes and ways we can minimise risk?

There's some irony here because despite how many times climate change has been mentioned in this thread the current reaction mirrors climate change discourse with the majority of the thread denying the possibility of real AI risks and not even considering it as an intellectual question.

Re: I resigned from Anthropic today

#720

Post by Findecanor »

icepush wrote: post #503
thomascountz wrote: post #456

Earlier quoted context omitted.

This is not correct. Please go through each one again, taking special note of global warming and nuclear weapon development .

Global warming is happening at a slow enough pace that there is plenty of time for people to move to the poles/into caves/beneath the ocean/whatever. Nuclear war would be much more destructive in a much shorter range of time but you will have pockets.

People already entrenched near the poles will fight to keep others out. Right-wing anti-immigrant parties are on the rise right now already.

They also tend to not want to recognise climate change -- because that would imply that people could migrate because of it.

Post reply on HN