For AI leaders Doom is a form of hype • Hacker News Board

Re: For AI leaders Doom is a form of hype

#91

Post by Applejack » Mon, Sep 14, 2026, 5:07 PM UTC

Isn't this just the big labs' latest move in the cycle to once again get regulation by their ignorant pals in the government(s) so that democratized open source models don't absolutely expose their over leveraging to secure and build compute and data centers.

...Of course at the same time they give lip-service to the importance of 'openness' for 'innovation' to try and placate individuals and even corporations that are sick of the rising costs.

Bottom-line, if they regulate mostly on the consumer side and then AI does get out of the lab (or a 'rogue' individual's rig) and starts wreaking havoc, are we supposed to be at the mercy of the big labs to summon their AI to come and save us plebs? I'm sure they won't stop to weigh whether that is in the interest of short term profits or Curtis Yarvin's techno-monarchy. /s

Isn't democratization the safest bet and what we should be demanding every time the Doom talk starts, whether it's coordinated or not?

Re: For AI leaders Doom is a form of hype

#92

Post by reducesuffering » Mon, Sep 14, 2026, 5:07 PM UTC

ACCount39 wrote: post #64

apsec112 wrote: post #23

All of the main characters here (Dario, Sam Altman, Demis Hassabis, Elon Musk) have been saying this for over ten years now, since before OpenAI or Anthropic even existed

This. They've been saying "AI is an extremely powerful, extremely dangerous technology" back when actual AIs were image classifiers that would struggle to tell a cupcake apart from a dog. OpenAI was founded by people who didn't trust Google to pursue AI tech responsibly. Anthropic was founded by people who didn't trust OpenAI to pursue AI tech responsibly. The "AI is incredibly dangerous" was the zeitgeist of AI labs…

Yep, anyone who thinks these leaders are only saying this now for marketing purposes hasn't even done a cursory 15 minute investigation into whether their beliefs are verifiably false.

Re: For AI leaders Doom is a form of hype

#93

Post by LogicFailsMe » Mon, Sep 14, 2026, 5:08 PM UTC

throwitaway222 wrote: post #70

LogicFailsMe wrote: post #58

Earlier quoted context omitted.

Stephen Hawking had superhuman intelligence. He never cured himself of ALS. Explain in great detail how a power-limited algorithm stuck in a datacenter takes over a world of humans with guns, missiles, and nukes. Nukes that are air-gapped with a human in the loop BTW. Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really,…

Non-stop pushing of narratives that stoke civil war?

That won't wipe us out, but it will cull the really violent ones along with a lot of noncombatants just like America's response to 9/11. Thank you, next?

Re: For AI leaders Doom is a form of hype

#94

Post by beepbooptheory » Mon, Sep 14, 2026, 5:08 PM UTC

atleastoptimal wrote: post #42

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits. This is an instance of work in a genre I call "AI la…

In your mind, what arguments about AI risk are ignored by these poems?

Re: For AI leaders Doom is a form of hype

#95

Post by LogicFailsMe » Mon, Sep 14, 2026, 5:08 PM UTC

frabcus wrote: post #69

LogicFailsMe wrote: post #58

Earlier quoted context omitted.

Stephen Hawking had superhuman intelligence. He never cured himself of ALS. Explain in great detail how a power-limited algorithm stuck in a datacenter takes over a world of humans with guns, missiles, and nukes. Nukes that are air-gapped with a human in the loop BTW. Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really,…

Stephen Hawking was a human.

Oh so now being an algorithm grants you magical powers?

Enumerate them.

Re: For AI leaders Doom is a form of hype

#96

Post by goatlover » Mon, Sep 14, 2026, 5:09 PM UTC

frabcus wrote: post #41

LogicFailsMe wrote: post #38

Earlier quoted context omitted.

Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

I think either "AI 2027" or "If Anyone Builds It, Everyone Dies" meet your criteria? I guess not literally, due to the dates? But they have substantive technical details.

Do the experts in the relevant fields agree with the "substantive" technical details on how the entire human race can be made extinct?

Re: For AI leaders Doom is a form of hype

#97

Post by fooker » Mon, Sep 14, 2026, 5:09 PM UTC

LogicFailsMe wrote: post #62

fooker wrote: post #11

Earlier quoted context omitted.

We don't really train deployed models in realtime in any realistic sense nowadays. It used to be a thing when ML was pretty much about classifying things into buckets.

May I introduce you to the word of our lord the arc-agi-3 kaggle competition? Because that's how they've hit 18% on a single RTX Pro 6000 with DIY RSI.

Where can I read more about this?

Re: For AI leaders Doom is a form of hype

#98

Post by [deleted] » Mon, Sep 14, 2026, 5:09 PM UTC

mrob wrote: post #71

mlsu wrote: post #57

Earlier quoted context omitted.

If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that th…

Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).

[deleted]

Re: For AI leaders Doom is a form of hype

#99

Post by LogicFailsMe » Mon, Sep 14, 2026, 5:09 PM UTC

sailfast wrote: post #74

LogicFailsMe wrote: post #38

Earlier quoted context omitted.

Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

Have you ever lived through a power outage longer than 48 hours? Did you see what it did to people in your neighborhood? That’s the path.

2 weeks, yes. Kids these days...

But today, we have ecoflows and solar panels. Back then, we didn't.

Re: For AI leaders Doom is a form of hype

#100

Post by frabcus » Mon, Sep 14, 2026, 5:10 PM UTC

LogicFailsMe wrote: post #83

frabcus wrote: post #41

Earlier quoted context omitted.

I think either "AI 2027" or "If Anyone Builds It, Everyone Dies" meet your criteria? I guess not literally, due to the dates? But they have substantive technical details.

No, they do not. They are both bullshit campfire ASI horror stories written by nontechnical sorts in the wishy washy invented field of AI safety. For example, I want someone with a proven track record in the field of robotics to explain the entire pathway from impressive 2026 Chinese robots to an entire self-assembly food chain from raw ore to von Neumann replicators that can travel to the stars and reproduce themsel…

Ha well, by definition what you want can't be provided!

Of course these stories can only be technical up until the recursive self-improvement part. Then they're necessarily fantasy, as the AI is more intelligent than the story author. Unfortunately, this doesn't make that scenario impossible.

However, we know now from Hugging Face that even without the later sci-fi powers, the LLMs are already capable of causing damage.

It doesn't need them to be able to make Von Neumann probes to e.g. steal their own weights from badly secured OpenAI servers, hack into various Neoclouds, distribute themselves, make Teslas crash into things, hack into all our power and water infrastructure and collapse global civilisation.