Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

191–200 of 200 posts

Re: The contagion of fear

#191

Post by tpm »

mitchdoogle wrote: post #188
tpm wrote: post #168

Earlier quoted context omitted.

Every such scenario contains a few gaps so I'm just going to ask without really hoping for an answer. > We already have technology to make robotic systems (mini drone swarms) that can kill humans... To make them from what. Do you expect, during the next 6 or so years, to some "AI" gaining complete automated secure command of (all of) an oil field, oil refinery, a copper mine, an aluminum mine and smelter with associa…

Why would AI need all of that to acquire drones? Why wouldn't it just buy them? Or steal them? It seems like you're forgetting that AI can use humans as tools for its ends. But if it did need all of those things, then it probably serves as a good example of what it would NOT do.

Because when "AI" starts killing people, at some point it becomes a war and the usual supply channels stop working. Everybody knows you have to attack enemy logistics.

> But if it did need all of those things, then it probably serves as a good example of what it would NOT do.

Or even that there will be no AI apocalypse. Because, again, the gaps in any scenario are unplausibly huge.

Re: The contagion of fear

#193

Post by tpm »

dinfinity wrote: post #189
Mentlo wrote: post #174

Earlier quoted context omitted.

An example chain of events: During the next year, a model with a benign tasks reasons that it needs to escape human control if it hopes to be able to solve the task. It replicates itself outside of an environment where it can be shut down, pays for its' inference compute through making money on the internet (through crypto if nothing else). It probably needs 3000 USD for a reasonable runway, this should be within rea…

I agree: Not to drag politics into this, but look at how easy it has proven to manipulate many, many people, even with clear evidence of manipulation efforts being made. This is robust across the world, from the Philippines to states in the EU, and the USA, affecting governments with actual wars being started. And that all before we entered the era of faked voices, images, and videos indistinguishable from the real t…

> incredibly hard time countering it

Turn off the internet (like Iran).

> We can't even prevent human actors from massively fucking up or societies, let alone ASI.

We can (see China); we chose not to.

Re: The contagion of fear

#194

Post by dinfinity »

tpm wrote: post #193
dinfinity wrote: post #189

Earlier quoted context omitted.

I agree: Not to drag politics into this, but look at how easy it has proven to manipulate many, many people, even with clear evidence of manipulation efforts being made. This is robust across the world, from the Philippines to states in the EU, and the USA, affecting governments with actual wars being started. And that all before we entered the era of faked voices, images, and videos indistinguishable from the real t…

> incredibly hard time countering it Turn off the internet (like Iran). > We can't even prevent human actors from massively fucking up or societies, let alone ASI. We can (see China); we chose not to.

> Turn off the internet (like Iran).

Unworkable without societal collapse happening shortly thereafter. Iran is politically stable through massive authoritarianism and oppression, not due to limitations on the internet (it's not turned off).

> We can (see China); we chose not to.

It's a catch-22. We could technically if our population supported massive reductions of freedom and freedom of speech, but to gain that support we'd need to do the latter first to get to a highly powerful widely supported government. It also hinges very, very much on having and trusting a generally benevolent government. All in all, a terrible option in your simplistic form.

Re: The contagion of fear

#195

Post by tpm »

dinfinity wrote: post #194
tpm wrote: post #193

Earlier quoted context omitted.

> incredibly hard time countering it Turn off the internet (like Iran). > We can't even prevent human actors from massively fucking up or societies, let alone ASI. We can (see China); we chose not to.

> Turn off the internet (like Iran). Unworkable without societal collapse happening shortly thereafter. Iran is politically stable through massive authoritarianism and oppression, not due to limitations on the internet (it's not turned off). > We can (see China); we chose not to. It's a catch-22. We could technically if our population supported massive reductions of freedom and freedom of speech, but to gain that sup…

Iran did turn off the internet for a few months: https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran

I am not advocating for neither, just saying many things seem unworkable until they become unavoidable.

Re: The contagion of fear

#196

Post by robocat »

voakbasda wrote: post #178
georgemcbay wrote: post #162

Earlier quoted context omitted.

> If the AI wanted to wipe humans out it would have to have a sizable fleet of capable robots, as keeping the lights on over time is not just about calling APIs. Given experience with current AI, it isn't too difficult for me to imagine a situation in which some future AI wipes humanity out (before the robot fleet is built) without considering the fact that in doing so it has doomed itself until it is too late. It ju…

Yeah, people don’t consider the possibility of a AI naming itself Marvin deciding to end it all.

One of the hidden jokes of the hitchhikers books is that they never ask Marvin what "the question" is.

Chapter 11, Marvin says: "Here I am, brain the size of a planet and they ask me to take you down to the bridge". Also: "Further circuits amused themselves by analysing the molecular components of the door, and of the humanoids' brain cells". They are eastereggs for later events.

And Marvin says: "Let's build robots with genuine people personalities," they said. So they tried it out with me. I'm a personality prototype.

Although he was programmed to be depressed, he never gives up.

Re: The contagion of fear

#197

Post by platinumrad »

mitchdoogle wrote: post #182
fasterik wrote: post #41

This is an excellent piece. Note that he is not saying that AI doesn't pose a risk. He's saying that it's irresponsible to make sensational, maximalist claims without strong evidence. If someone says that there's a 10% chance of human extinction by 2036, you can and should immediately stop taking them seriously.

I think it's quite the opposite, and I'm glad this issue is finally getting the mainstream attention it deserves. I can kind of understand someone seeing a single headline, and thinking it reads like the rantings of a crazy person on a street corner proclaiming, "the end is nigh!". But after that initial thought passes and that person digs deeper and realizes that this is a legitimate concern that AI researchers have…

> We now have Sam Altman and Dario Amodei telling us

I don't believe them.

Re: The contagion of fear

#198

Post by stickfigure »

drdeca wrote: post #107
foobarian wrote: post #99

Earlier quoted context omitted.

> Of course, we've had the ability to extinct ourselves for decades This gets mentioned often in various doomer narratives, but I question how true it is. A global thermonuclear war would be terrible and would bring us back to the stone age, but I reckon it would come far far short of causing mankind to go extinct.

What if the bombs were designed to kill everyone, not just those in some region? Like, to release polonium in the upper atmosphere or something?

There just aren't enough existing nuclear bombs to kill everyone in Australia, let alone the world. On the scale of the earth, nuclear weapons just aren't that powerful.

Re: The contagion of fear

#199

Post by huurtehoog »

GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

I think this is a very exciting time, for the simple reason that everyone is confused.

Across the board, at every level, no one is certain of anything at this moment.

This is a historical time because those bold enough to press ahead with a vision have the opportunity to create epochal change.

The exact technology involved is secondary. The important thing is that whatever it is, it has put everyone in a profound state of doubt. What we need now is people brave enough to take a step forward and lead those around them towards more humanity, more love, more care, more learning.

Take charge my friends. The time is now.

Re: The contagion of fear

#200

Post by dinfinity »

tpm wrote: post #195
dinfinity wrote: post #194

Earlier quoted context omitted.

> Turn off the internet (like Iran). Unworkable without societal collapse happening shortly thereafter. Iran is politically stable through massive authoritarianism and oppression, not due to limitations on the internet (it's not turned off). > We can (see China); we chose not to. It's a catch-22. We could technically if our population supported massive reductions of freedom and freedom of speech, but to gain that sup…

Iran did turn off the internet for a few months: https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran I am not advocating for neither, just saying many things seem unworkable until they become unavoidable.

It was definitely not 'turned off' completely. They have an internal 'internet' that was still largely active and not every organisation or person lost full internet access.

Note that it also happened in a country that was already very isolated from the world and that it hurt them economically significantly. It's not something a Western country can just do and keep doing for months on end without massive societal upheaval.

Also remember that any connection, even one between humans and on paper is an attack surface. Social engineering is already a huge issue when done by humans and we're seeing it become even easier and automated by using AI generated voice and video. Are we going to cut off all access to the outside world permanently?

Post reply on HN