For AI leaders Doom is a form of hype • Hacker News Board

Re: For AI leaders Doom is a form of hype

#51

Post by SpicyLemonZest » Mon, Sep 14, 2026, 4:51 PM UTC

LogicFailsMe wrote: post #38

palmotea wrote: post #30

Earlier quoted context omitted.

> The timing of the AI doom narrative could not come as a more suspect time....There's little hope investors could recoup the wild investments made into Anthropic and OpenAI Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.

Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

If someone showed you such a path, would you consider changing your mind? You're always going to be able to find a reason that such an explanation doesn't count if you're dedicated to looking for one.

Re: For AI leaders Doom is a form of hype

#52

Post by shimman » Mon, Sep 14, 2026, 4:51 PM UTC

schainks wrote: post #36

The cult-like behavior we're seeing from these "AI Leaders": 1. OMG our technology could destroy the world, watch out! 2. Only one person can save you 3. IT ME Unfortunately, the governance layer meant to put a foot in the butt of this behavior is unavailable. This week, we woke up to clear signs The Bad Guys have been using tech from our innovator AI companies. Rather than these AI leaders admitting they clearly had…

Yeah, this is how fascists talk and with seeing how this technology has done nothing to improve society (in fact all it seems to be doing is immiserating workers, enabling a surveillance society, and quite useful in helping target children at schools for drone strikes) it's not hard to be not only immediately skeptical but to resist it in the most extreme forms.

This also seems to be their answer against the data center backlash, SF + VC are quite stupid at politics and I don't think they understand how extreme Americans are willing to go to protect their way of life.

Re: For AI leaders Doom is a form of hype

#53

Post by svara » Mon, Sep 14, 2026, 4:51 PM UTC

LogicFailsMe wrote: post #38

palmotea wrote: post #30

Earlier quoted context omitted.

> The timing of the AI doom narrative could not come as a more suspect time....There's little hope investors could recoup the wild investments made into Anthropic and OpenAI Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.

Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

I don't think we'd be okay with that if it took a few years longer, right?

I thought the Bengio blog post posted here the other day answered that question quite well (ignoring your timeline).

The argument is that in training for the capability to achieve certain goals, you may inadvertently also train for secondary instrumental goals you did not intend, such as aggressive behavior or deception.

I'm actually no sure I buy that this is particularly likely, or necessary, but I found that take reasonable and worth considering.

Re: For AI leaders Doom is a form of hype

#54

Post by apsec112 » Mon, Sep 14, 2026, 4:52 PM UTC

The sources that this article cites have aged very badly and the author doesn't seem to have noticed this

"But, just as happened with nanotech, the wind appears to be going out of the sales of “AI.” Some researchers suggest that we may be entering a new “AI Winter,” a period of decreased funding in the area, or at least an “AI Autumn,” as exuberance for the technology fades and expectations come back to earth." (written in 2021! from the cited Lee Vinsel Medium article)

"ChatGPT is nothing more than souped-up autocomplete, [so] why are so many people convinced that it’s actually “understanding” and “reasoning”?" - the cited Emily M. Bender book, written last year

Re: For AI leaders Doom is a form of hype

#55

Post by goatlover » Mon, Sep 14, 2026, 4:52 PM UTC

atleastoptimal wrote: post #42

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits. This is an instance of work in a genre I call "AI la…

> they believe only they can be stewards of digital gods

You do't see how absurd that sounds? Not only the idea that they can create gods, but that they are the only good stewards for such beings. It's also incredibly arrogant.

Re: For AI leaders Doom is a form of hype

#56

Post by phoghed » Mon, Sep 14, 2026, 4:52 PM UTC

atleastoptimal wrote: post #42

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits. This is an instance of work in a genre I call "AI la…

The author specifically calls out that the analysis is done by perplexity

Re: For AI leaders Doom is a form of hype

#57

Post by mlsu » Mon, Sep 14, 2026, 4:52 PM UTC

atleastoptimal wrote: post #42

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits. This is an instance of work in a genre I call "AI la…

If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that thing. And yet, here's Fable 6, more capable than ever before, oh BTW we are IPOing, oh, BTW, I have stock options...

Re: For AI leaders Doom is a form of hype

#58

Post by LogicFailsMe » Mon, Sep 14, 2026, 4:52 PM UTC

frabcus wrote: post #28

swingboy wrote: post #6

Can anyone from a lab anonymously confirm they have or are insanely close to real-time weight modifications and still using the same architecture every other lab is using (transformers)?

If it helps, this person from OpenAI explains at least something: "I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly." https://x.com/MajmudarAdam/status/2098881885200081234 The explanation is basically they have other dimensions (not just pretraining and inference compute) that scale, and they've got fairly convincing scaling laws. And they know they ca…

Stephen Hawking had superhuman intelligence. He never cured himself of ALS. Explain in great detail how a power-limited algorithm stuck in a datacenter takes over a world of humans with guns, missiles, and nukes. Nukes that are air-gapped with a human in the loop BTW.

Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really, what happens?

To that end, all we ever get is nontechnical hand-waving about curing cancer, immortality, and von Neumann replicators and then the ASI somehow wipes us out but how? And don't you dare say by designing a chemical weapon or bio agent without spelling out the entire process step by $%^#ing step because details matter. It's gonna do superpersuasion, sure, but have you ever heard of komprimat? There is nothing new under the sun here.

Edit: believing in AI 2027 is every bit as cray cray as believing in the rapture. Both require an insane leap of faith to reach their final conclusions.

Re: For AI leaders Doom is a form of hype

#59

Post by palmotea » Mon, Sep 14, 2026, 4:52 PM UTC

LogicFailsMe wrote: post #38

palmotea wrote: post #30

Earlier quoted context omitted.

> The timing of the AI doom narrative could not come as a more suspect time....There's little hope investors could recoup the wild investments made into Anthropic and OpenAI Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.

Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

> Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

Why? Those aren't the things I'm actually worried about, and those aren't the only "doomer" ideas, just the most extreme ones (thus easy to straw-man).

My main worries are about economic disruption, social damage due to the pace of technological change outpacing human ability to manage it, further concentration of power into small elites (along existing lines), and the reallocation of labor towards less personally-fulfilling pursuits.

I also totally buy into the idea that "AI leaders" are playing up certain risks as a marketing strategy. After all these are smart tech people, and those types have a tendency to get really arrogant and be irresponsible.

So the main thing is to slow all this down, so there's a chance of getting it under control, and snatch the payday away from such greedy and irresponsible people.

Re: For AI leaders Doom is a form of hype

#60

Post by carodgers » Mon, Sep 14, 2026, 4:53 PM UTC

There's been substantial effort put into designing and running benchmarks for these models. You know which one they don't report any more? Chess ELO.

Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess.

Lecun has the correct approach. Mock these people relentlessly for their attention-seeking doomerism.