Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

131–140 of 200 posts

Re: The contagion of fear

#131

Post by pstuart »

shimman wrote: post #112
foobarian wrote: post #99

Earlier quoted context omitted.

> Of course, we've had the ability to extinct ourselves for decades This gets mentioned often in various doomer narratives, but I question how true it is. A global thermonuclear war would be terrible and would bring us back to the stone age, but I reckon it would come far far short of causing mankind to go extinct.

I mean it's likely a nuclear war would probably be a species extinction level event at the least, humans would definitely be reduced by 99%. We don't know how bad the resulting nuclear cooling would be, but we do know how humans act under extreme desperation. They lash out and attack, when was the last time a majority of human settles were truly under desperate acts of survival simultaneously? Maybe what ~74,000 year…

If the AI wanted to wipe humans out it would have to have a sizable fleet of capable robots, as keeping the lights on over time is not just about calling APIs.

Re: The contagion of fear

#132

Post by dullcrisp »

cowthulhu wrote: post #129
dullcrisp wrote: post #127

Earlier quoted context omitted.

Yes, just like that. What would it take for you to accept Our Lord into your heart?

The bar is high, but there's probably some combination of events (combined with a high confidence in my own lucidity over an extended period of time) that would convince me.

Same same.

Re: The contagion of fear

#133

Post by hn_throwaway_99 »

fasterik wrote: post #87
hn_throwaway_99 wrote: post #82

Earlier quoted context omitted.

Well, I'll have to disagree that this is an excellent piece, but that's another issue. And I do agree that AI killing all humans by 2036 doesn't appear plausible to me. But what is plausible is we could easily be down a path so that by 2036 "future doom" already is a very likely risk. All of the frontier AI companies have been racing to automate themselves, that is, where AI fully autonomously build the next generati…

None of that has anything to do with the article, and if you thought the message was "this is overhyped bullshit" then you should go back and read it again. As I already pointed out, he isn't saying anything about the probablity of harms or disasters from AI. He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arg…

There have been measured claims backed up by arguments and evidence. My frustration is that people aren't addressing the specific arguments that have been made:

1. https://www.aifutures.org/ outlines a number of specific scenarios, and importantly details their methodology for each.

2. Independent researchers in the Hugging Face incident outlined how previously predicted misalignment scenarios actually played out, and outlined how slightly more advanced AI, or slightly more misaligned, or with more access to critical infrastructure, could cause immense harm: https://www.planned-obsolescence.org/p/the-hugging-face-atta...

3. Technical leaders at OpenAI (specifically their chief scientist) outlined the problems they gave with controlling models now: https://openai.com/index/an-alien-mind/

None of the specific arguments in these or many other detailed explanations of how an AI takeover could occur were even acknowledged.

Re: The contagion of fear

#135

Post by hn_throwaway_99 »

ElProlactin wrote: post #125
fasterik wrote: post #87

Earlier quoted context omitted.

None of that has anything to do with the article, and if you thought the message was "this is overhyped bullshit" then you should go back and read it again. As I already pointed out, he isn't saying anything about the probablity of harms or disasters from AI. He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arg…

> He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arguments and evidence. What he's asking for isn't possible in the form he's asking for it. AI experts can't even agree on what AI is , what it's capable of and what the limits of its development are. If the experts can't even agree on what's happening "inside…

> > AI executes on physical systems that have been engineered with human accountability and control. Intelligence does not exempt a system from the realities of the physical world!

> For someone making a point about responsibility, this is ridiculously irresponsible.

It's not just irresponsible, it's false. Russia killed 3 Ukrainian civilians with a drone where the targeting was completely autonomous by AI running on an Nvidia chip: https://www.nytimes.com/2026/08/24/world/europe/russia-drone.... The Pentagon tried to completely blacklist Anthropic because Anthropic refused to allow autonomous kills without a human in the loop. If you can't see how lots of military leaders want to put more lethal control into AI at this point I think you have to be willfully blind.

Re: The contagion of fear

#136

Post by ElProlactin »

mplanchard wrote: post #130
ElProlactin wrote: post #125

Earlier quoted context omitted.

> He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arguments and evidence. What he's asking for isn't possible in the form he's asking for it. AI experts can't even agree on what AI is , what it's capable of and what the limits of its development are. If the experts can't even agree on what's happening "inside…

> If the experts can't even agree on what's happening "inside of" these LLMs, how can they give laypeople an assessment of the risk? The answer, if you are a responsible expert in the field, is to convey the range of possibilities and the uncertainty.

Isn't that what they're doing? "There's a better than 0 chance this thing could kill everyone in the next decade."

That might be too imprecise for the HN set but it's realistic for laypeople.

And none of the AI people talking about the risk are running into rooms full of people telling them Claude has gone mad and yelling at them to disconnect from the internet and turn off their devices immediately.

Re: The contagion of fear

#137

Post by fasterik »

hn_throwaway_99 wrote: post #133
fasterik wrote: post #87

Earlier quoted context omitted.

None of that has anything to do with the article, and if you thought the message was "this is overhyped bullshit" then you should go back and read it again. As I already pointed out, he isn't saying anything about the probablity of harms or disasters from AI. He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arg…

There have been measured claims backed up by arguments and evidence. My frustration is that people aren't addressing the specific arguments that have been made: 1. https://www.aifutures.org/ outlines a number of specific scenarios, and importantly details their methodology for each. 2. Independent researchers in the Hugging Face incident outlined how previously predicted misalignment scenarios actually played out, an…

None of those are arguments giving a probability of human extinction before 2036 or any other time. AI Futures has said that some members of their team believe that it's 10-30% at some unspecified point in the future, but there is no justification given for where those numbers come from.

Re: The contagion of fear

#138

Post by besterman23 »

ares623 wrote: post #102

I might be alone in this, but 10% chance of humanity being wiped out in the next 10 years is a small price to pay if it means I can avoid writing YAML config by hand ever again. Look, I love my kids and all, but c'mon... 30+ years more of YAML? I think they'll understand.

Honestly I think it’s an apt punishment for ever inventing YAML in the first place.

Re: The contagion of fear

#139

Post by dullcrisp »

sobellian wrote: post #70

Any claim that AI will, or could, destroy humanity reduces to a claim that any sufficiently intelligent being - even a human - could destroy humanity. I find that much of the x-risk thought relies on religious thinking. Take for example: https://x.com/paulg/status/1660404244174782464 . If this line of thinking is taken too literally, we can never falsify it. Any specific hypothesis - nukes, bioweapons, spontaneously…

Have we quantified the x-risk of Magnus Carlsen?

Re: The contagion of fear

#140

Post by webern777 »

SamBam wrote: post #38
jchw wrote: post #28

Edit: Actually, nevermind. You guys are not capable of having a reasonable discussion about this.

> The problem isn't that AI could never be capable of causing real world damage. It is that there is absolutely no fucking path towards human extinction. I think the "humanity will go extinct" discussion is a silly strawman being propagated either to try to spread fear, as the article suggests, or to make the anti-AI folks look silly, but it detracts from your first point which is that it may be "capable of causing r…

Or you could argue that the super intelligence will turn us into a pet of some kind. Then the risk is we think we are controlling it while we are really the super intelligence's lap dog.

That is an argument that at least makes sense.

The super intelligence as homicidal maniac just doesn't make sense. There is less intelligent wildlife all around us and we mostly completely ignore it. We have more interesting things to do with our time as intelligent beings than carry out a bird or rabbit genocide.

It almost seems like the projection of some kind of paranoid delusion about change.

Post reply on HN