Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For AI leaders Doom is a form of hype

erkansaka.net

Post reply on HN
Print view View original

81–90 of 172 posts

Re: For AI leaders Doom is a form of hype

#81

Post by mlsu »

aaroninsf wrote: post #77
mlsu wrote: post #57

Earlier quoted context omitted.

If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that th…

This is not an argument, it's an unfounded claim based on a straw argument. The threat is distributed; the problem is systemic. Social herd behavior is very real, and it's a probability space, not a certainty. Similar case: climate change is without any question causing millions of deaths per year and may well kill billions, not to mention destroying ecosystems, and how may people do you know who have given up eating…

You're right. That's why I'm doing my best to give the oil companies more power and influence. They are, after all, in the best position to stop climate change!

Re: For AI leaders Doom is a form of hype

#82

Post by shimman »

throwitaway222 wrote: post #75
shimman wrote: post #72

Earlier quoted context omitted.

Why? I was raised as an evangelical christian. End of time sermons are quite common, along with telling followers that they are warriors of christ. This doesn't seem any different than what these MBA leaders are saying "AI will destroy humanity, I can protect it." It's delusional but it's quite effective, luckily it doesn't work on the general population but you can get a dedicated group of people, and as we see with…

Just stay away from it - if @dang allows it, then someone else will justifiably start saying "on par with shia muslims" in the same way you wrote your text.

I guess but I don't see how it wouldn't be allowed. Evangelicals have quite a stranglehold on US politics and have been influencing the republican party for like 40 years now. Trump

Not being allowed to thread the connections seems odd, it's how we know what these same leaders are going to do next.

Re: For AI leaders Doom is a form of hype

#83

Post by LogicFailsMe »

frabcus wrote: post #41
LogicFailsMe wrote: post #38

Earlier quoted context omitted.

Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

I think either "AI 2027" or "If Anyone Builds It, Everyone Dies" meet your criteria? I guess not literally, due to the dates? But they have substantive technical details.

No, they do not. They are both bullshit campfire ASI horror stories written by nontechnical sorts in the wishy washy invented field of AI safety. For example, I want someone with a proven track record in the field of robotics to explain the entire pathway from impressive 2026 Chinese robots to an entire self-assembly food chain from raw ore to von Neumann replicators that can travel to the stars and reproduce themselves, not just assert that ASI will make that happen among so many other unlikely events like curing cancer because reasons because all any of this takes is superintelligence according to these sorts.

https://garymarcus.substack.com/p/the-ai-2027-scenario-how-r...

But no one likes hearing from the engineers on this, I get it. We are simply no fun. So many would rather believe a time traveling superintelligence will punish a simulated avatar of them for not making the ASI happen sooner. And even if that were true, why am I supposed to care about what happens to a simulated avatar of me?

None of this makes any logical sense whatsoever.

Re: For AI leaders Doom is a form of hype

#84

Post by frabcus »

mlsu wrote: post #81
aaroninsf wrote: post #77

Earlier quoted context omitted.

This is not an argument, it's an unfounded claim based on a straw argument. The threat is distributed; the problem is systemic. Social herd behavior is very real, and it's a probability space, not a certainty. Similar case: climate change is without any question causing millions of deaths per year and may well kill billions, not to mention destroying ecosystems, and how may people do you know who have given up eating…

You're right. That's why I'm doing my best to give the oil companies more power and influence. They are, after all, in the best position to stop climate change!

You joke, but alas...

"Fossil fuel lobbyists flood COP30 climate talks in Brazil, with largest ever attendance share" https://kickbigpollutersout.org/Release-Kick-Out-The-Suits-C...

Re: For AI leaders Doom is a form of hype

#85

Post by reaperducer »

NoahZuniga wrote: post #19

> I do not doubt that these people are sincere in some private, psychological sense ... . As a media scholar, what interests me is not whether Hubinger believes his own number. What interests me is what this genre of utterance does in the world ... . My complaint is not that these statements are alarming. It is that they are apocalyptic in form, hypocritical in structure, and strategically productive in effect — and…

Like if its true it can't be hype

I was kind of leaning toward thinking all the talk of doom was a bunch of self-serving regulatory-capture hype until this morning when the New York Times ran a long story about the Chinese military is also saying that AI is a threat.

If two sides so far apart from one another in so many ways see the same thing, there must be something to it.

https://www.nytimes.com/2026/09/14/world/asia/china-ai-secur...

Re: For AI leaders Doom is a form of hype

#86

Post by reducesuffering »

ianm218 wrote: post #25

What I don't really understand with these posts is that they don't at any point contend with the "doomers" argument. Like people get so caught in people acting hypocritical/ having mixed incentives/ or just in many cases personally disliking West Coast billionaire types and the kind of employees they hire. All of which seems fair to point out. But like the core argument - AI is getting better at core research tasks (…

Because they can't. If they actually engaged with it, they would realize how shallow the "doomer" / existential risk dismissals are. Meanwhile there are ample essays and arguments about the existential issues from Turing award winners, frontier lab CEO's, and many more. There's also https://ai-2040.com/ , you will never see the equivalent on the other side of the debate because it won't hold under any scrutiny.

Re: For AI leaders Doom is a form of hype

#87

Post by Gregkion »

carodgers wrote: post #60

There's been substantial effort put into designing and running benchmarks for these models. You know which one they don't report any more? Chess ELO. Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess. Lecun has the correct approach. Mock the…

It doesn't really matter if it will be an LLM or Lecuns world model. Lecuns world model, if its the right architecture, will be retrained by google, anthropic and co with the massive amount of compute they have and the massive amount of data.

Also i'm pretty sure Lecun didn't get his money from thin air so big companies are invested in this one way or the other.

And an LLM can easily, today, just rebuild a chess engine and beat whatever they need to beat. It doesn't make sense to ask a random LLM for a next chess move. Would you ask a random human to beat a chess master?

We have reached peak learning. Its probalby now cheaper to teach 1 LLM something new than teaching it to 100.000 people.

Blender support got a lot better just this month. So much better that its quesitonable if anyone new/young wants to learn blender today.

Re: For AI leaders Doom is a form of hype

#88

Post by conmod278 »

NoahZuniga wrote: post #19

> I do not doubt that these people are sincere in some private, psychological sense ... . As a media scholar, what interests me is not whether Hubinger believes his own number. What interests me is what this genre of utterance does in the world ... . My complaint is not that these statements are alarming. It is that they are apocalyptic in form, hypocritical in structure, and strategically productive in effect — and…

As Von Neumann said about Oppenheimer's performative angst over the atomic bomb:

"Some people profess guilt to claim credit for the sin."

Re: For AI leaders Doom is a form of hype

#89

Post by LogicFailsMe »

frabcus wrote: post #78
LogicFailsMe wrote: post #58

Earlier quoted context omitted.

Stephen Hawking had superhuman intelligence. He never cured himself of ALS. Explain in great detail how a power-limited algorithm stuck in a datacenter takes over a world of humans with guns, missiles, and nukes. Nukes that are air-gapped with a human in the loop BTW. Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really,…

Hacks in to neoclouds for more compute (e.g. like Hugging Face incident), socially manipulates people to do things for them (e.g. like social media recommendation algorithms), hacks into lab's own training/monitoring/inference (e.g. as swarm did into OpenAIs eval cluster). See "AI 2027" or "If Anyone Builds It, Everyone Dies" for some more ideas.

Cool, AWS falls to the ASI, oh no!

But how does it make the fundamental breakthroughs to &%^$ing von Neumann replicators that can reproduce themselves from raw materials harvested from nearby solar systems? I'll wait. Because without this breakthrough, the ASI won't get its robot army either.

I can absolutely see a rogue ASI though. But unless it radically improves power efficiency, we can just shut down the power to its datacenters, by force if necessary. And then we painfully repair the resiliency of our infrastructure by finally being relieved of the option of ignoring it.

Re: For AI leaders Doom is a form of hype

#90

Post by goatlover »

mrob wrote: post #71
mlsu wrote: post #57

Earlier quoted context omitted.

If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that th…

Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).

This sounds like a rationalization for gambling with everyone's lives, assuming they really believe such a thing.
Post reply on HN