Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For AI leaders Doom is a form of hype

erkansaka.net

Post reply on HN
Print view View original

111–120 of 167 posts

Re: For AI leaders Doom is a form of hype

#111

Post by mrob »

bigstrat2003 wrote: post #109
mrob wrote: post #71

Earlier quoted context omitted.

Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).

> If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. That is completely irrational.

1% chance of survival beats 0%.

Re: For AI leaders Doom is a form of hype

#112

Post by LogicFailsMe »

frabcus wrote: post #100
LogicFailsMe wrote: post #83

Earlier quoted context omitted.

No, they do not. They are both bullshit campfire ASI horror stories written by nontechnical sorts in the wishy washy invented field of AI safety. For example, I want someone with a proven track record in the field of robotics to explain the entire pathway from impressive 2026 Chinese robots to an entire self-assembly food chain from raw ore to von Neumann replicators that can travel to the stars and reproduce themsel…

Ha well, by definition what you want can't be provided! Of course these stories can only be technical up until the recursive self-improvement part. Then they're necessarily fantasy, as the AI is more intelligent than the story author. Unfortunately, this doesn't make that scenario impossible. However, we know now from Hugging Face that even without the later sci-fi powers, the LLMs are already capable of causing dama…

Explain how rogue FSD cars take down civilization. And I say that with the opinion that the only good scenes in The Fate of The Furious and Leave The World Behind were the ones with the rogue Teslas. But also, nothing new under the sun really:

https://en.wikipedia.org/wiki/Maximum_Overdrive

And so granted we could probably get a global 9/11 in deeply urban regions out of your scenario, we then bomb the datacenters and we finally address the horrific tech debt in our infrastructure blissfully relieved of the option of continuing to ignore it. Don't think this is happening either BTW but I acknowledge its probability is slightly greater than zero.

To that end, my house is entirely off-grid. And in the event of what you call the end of civilization but I call the big burp inconvenience, I've made friends with my neighbors because community is what really matters during a disaster, and I've been through a few.

Re: For AI leaders Doom is a form of hype

#113

Post by [deleted] »

Gregkion wrote: post #87
carodgers wrote: post #60

There's been substantial effort put into designing and running benchmarks for these models. You know which one they don't report any more? Chess ELO. Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess. Lecun has the correct approach. Mock the…

It doesn't really matter if it will be an LLM or Lecuns world model. Lecuns world model, if its the right architecture, will be retrained by google, anthropic and co with the massive amount of compute they have and the massive amount of data. Also i'm pretty sure Lecun didn't get his money from thin air so big companies are invested in this one way or the other. And an LLM can easily, today, just rebuild a chess engi…

[deleted]

Re: For AI leaders Doom is a form of hype

#114

Post by LogicFailsMe »

goatlover wrote: post #96
frabcus wrote: post #41

Earlier quoted context omitted.

I think either "AI 2027" or "If Anyone Builds It, Everyone Dies" meet your criteria? I guess not literally, due to the dates? But they have substantive technical details.

Do the experts in the relevant fields agree with the "substantive" technical details on how the entire human race can be made extinct?

Have you noticed they're not interviewing these people? IMO that would totally kill the AI 2027 buzz.

Re: For AI leaders Doom is a form of hype

#115

Post by LogicFailsMe »

Gregkion wrote: post #107
LogicFailsMe wrote: post #104

Earlier quoted context omitted.

Actually, some sorts do assert humanity will be wiped off the planet by AI. I don't share that view either. I think it's ludicrous. We've lived through much worse without any sort of technology beyond fire and a big stick. But the end of scarcity and disease is a fantastic thing if it happens. If our lizard brains do not bridge that transition and instead attempt to maintain the billionaire status quo, I agree, we're…

Musk did a 180 degree turn around. Killing USAID etc. i don't think he sees normal humans which do not help him or are not aligned to him as humans anymore. And Zuckerberg bought so much land and bunkers, he is prepared for shit hitting the fan. After all he doesn't care what his platform did to humanity anyway.

And we have a planet full of general contractors who know their way around welding and pouring concrete down ventilation shafts. Many of whom these sorts wanted deported. If they want to build a Per Stenson terrarium and flee into it, by all means do so. And then let's seal them inside, problem solved.

Re: For AI leaders Doom is a form of hype

#116

Post by atleastoptimal »

beepbooptheory wrote: post #94
atleastoptimal wrote: post #42

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits. This is an instance of work in a genre I call "AI la…

In your mind, what arguments about AI risk are ignored by these poems?

Many posit that ASI or rogue AI agents are impossible, a fantasy, or marketing hype. Many claimed that the entire HuggingFace incident was an invented conspiracy made up by OpenAI and HF.

The most common issue I have with them is their insistence on rigid classification: "AI is a tool, it doesn't have intentions, or desires, only humans do", as if that rhetorical flourish eliminates years of alignment and capabilities research. There is a desire to characterize all the developments in LLM's as some sort of parlor trick which by definition cannot be greater than the sum of its parts (i.e. the outputs of an LLM can never be greater than the sum of its training data, genuine artificial "intelligence" is impossible, etc)

Re: For AI leaders Doom is a form of hype

#117

Post by raddan »

matherial wrote: post #68
apsec112 wrote: post #54

The sources that this article cites have aged very badly and the author doesn't seem to have noticed this "But, just as happened with nanotech, the wind appears to be going out of the sales of “AI.” Some researchers suggest that we may be entering a new “AI Winter,” a period of decreased funding in the area, or at least an “AI Autumn,” as exuberance for the technology fades and expectations come back to earth." (writ…

The author appears to be an LLM. This seems to be a 100% AI-generated article, which makes the "as a media scholar" passage especially hilarious.

If the author is an LLM, it’s a really bad one.

> wind appears to be going out of the sales of “AI.”

is a sentence that virtually no AI would write since the most probable word here is “sails.” I can’t read the article though since it appears that the site is down.

I suspect that it’s just a bad/dated analysis.

Re: For AI leaders Doom is a form of hype

#118

Post by LogicFailsMe »

fooker wrote: post #97
LogicFailsMe wrote: post #62

Earlier quoted context omitted.

May I introduce you to the word of our lord the arc-agi-3 kaggle competition? Because that's how they've hit 18% on a single RTX Pro 6000 with DIY RSI.

Where can I read more about this?

https://www.kaggle.com/competitions/arc-prize-2026-arc-agi-3

Re: For AI leaders Doom is a form of hype

#119

Post by SpicyLemonZest »

LogicFailsMe wrote: post #106
SpicyLemonZest wrote: post #51

Earlier quoted context omitted.

If someone showed you such a path, would you consider changing your mind? You're always going to be able to find a reason that such an explanation doesn't count if you're dedicated to looking for one.

Yes, actually I would. There's a long list of people in my career path who bet on me sticking to my guns against compelling evidence to the contrary. But I know, I know, that's impossible! Anyway, after they figured out I would change my mind, then they described me as lacking conviction because you can't win, but I digress. But upfront, it's a detailed path with specific breakthroughs and a concrete plan to achieve…

I've read Gary Marcus's critique of AI 2027. How do you grapple with the fact that superhuman coding arrived on schedule, despite his skepticism that it would? Doesn't that mean that the other predictions are also more plausible than we may think?

Re: For AI leaders Doom is a form of hype

#120

Post by carodgers »

Gregkion wrote: post #87
carodgers wrote: post #60

There's been substantial effort put into designing and running benchmarks for these models. You know which one they don't report any more? Chess ELO. Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess. Lecun has the correct approach. Mock the…

It doesn't really matter if it will be an LLM or Lecuns world model. Lecuns world model, if its the right architecture, will be retrained by google, anthropic and co with the massive amount of compute they have and the massive amount of data. Also i'm pretty sure Lecun didn't get his money from thin air so big companies are invested in this one way or the other. And an LLM can easily, today, just rebuild a chess engi…

"And an LLM can easily, today, just rebuild a chess engine and beat whatever they need to beat."

Always claims and promises about what they "can" do. This pattern is constant across AI hype. No one can prove that an LLM will forever fail to solve problem X, so the claim goes uncontested, and humans are biased to believe unrebutted claims.

I reject this fallacious line of reasoning. I will instead require evidence of what they "have" done. No LLM to date has written a chess engine that beats stockfish. More importantly, no LLM has ever written any chess engine without heavily copying code and patterns from humans.

Post reply on HN