Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For AI leaders Doom is a form of hype

erkansaka.net

Post reply on HN
Print view View original

151–160 of 172 posts

Re: For AI leaders Doom is a form of hype

#151

Post by ahf8Aithaex7Nai »

pixl97 wrote: post #144
ahf8Aithaex7Nai wrote: post #127

Earlier quoted context omitted.

> Just because you list up some type of random technology, doesn't make it an good argument. In what way is this random? It's a chronology of the bullshit from the past few years. Also: easy to say in hindsight. The hype was there, and many people believed it. The press, in particular, just lapped up the claims the way a Labrador Retriever gobbles up whatever you put in front of it. > We have opened up a potential pa…

Hey, I remember when the Internet was just a bunch of bullshit that was going to go away too. >It has no consciousness. Why does it need it? >It has no intentions. All it needs is a prompt. Instrumental convergence (and humans hooking them up to everything else) takes care of the rest. Really you sounds horrifically ignorant of how any evolutionary AI algorithms work and the problems we see come out of them in much s…

Yeah, yeah, sure, I'm ignorant. In any case, I recognize unfalsifiability when I see it. So it seems there's still some room for even more ignorance beneath my ignorance.

Re: For AI leaders Doom is a form of hype

#152

Post by LogicFailsMe »

palmotea wrote: post #134
LogicFailsMe wrote: post #83

Earlier quoted context omitted.

No, they do not. They are both bullshit campfire ASI horror stories written by nontechnical sorts in the wishy washy invented field of AI safety. For example, I want someone with a proven track record in the field of robotics to explain the entire pathway from impressive 2026 Chinese robots to an entire self-assembly food chain from raw ore to von Neumann replicators that can travel to the stars and reproduce themsel…

> None of this makes any logical sense whatsoever. It doesn't make any logical sense to set the bar as as you have, either. I mean if someone could provide the roadmap you're asking for, it's probably already too late. And while we're talking about sci-fi, it seems fairly likely that an AI that could "kill us all" could fail to build a "entire self-assembly food chain from raw ore to von Neumann replicators." I'd be…

Fort Detrick and likely its Russian, Chinese, and possibly Indian equivalents have had stocks of bio agents that could give a humanity a very bad day for decades. I'm far more worried about those existing stocks than I am about a tech nerd trying to DIY the genocidal plague.

Which is to say it's already a threat, it's unproven but not entirely impossible COVID leaked from a lab as just one example, but Ebola suggests that it requires more than a rogue nerd to spread a deadly virus global because once people start dying in the streets, the rest hide because they don't want to die. COVID spread because it wasn't deadly to 99% of humanity and now most of the 1% vulnerable to it are either dead or permanently crippled by long COVID so it's ambient now.

I worry about attacks to the infrastructure, but I just don't buy that as a direct route to civilizational collapse.

Re: For AI leaders Doom is a form of hype

#153

Post by reaperducer »

jackb4040 wrote: post #121
reaperducer wrote: post #85

Earlier quoted context omitted.

Like if its true it can't be hype I was kind of leaning toward thinking all the talk of doom was a bunch of self-serving regulatory-capture hype until this morning when the New York Times ran a long story about the Chinese military is also saying that AI is a threat. If two sides so far apart from one another in so many ways see the same thing, there must be something to it. https://www.nytimes.com/2026/09/14/world/a…

https://archive.ph/BzBjZ Read the article. The things the Chinese officials are worried about are: * Fake videos / disinformation * Cyberattacks on critical infrastructure * Data leaks * Advances in military sensing / targeting technology These concerns have little in common with the Yudkowskian / apocalyptic doom marketing the US hyperscalers are pushing now. The Times even says this explicitly in the second paragra…

Read the article.

I did, thanks. I don't need you to tell me to do that.

The common thread is that they both see AI as a threat. It's a different threat to each side, but still a threat.

Just because they see different threats from AI doesn't mean that AI isn't a threat.

If one person wants to paint the house red and another person wants to paint the house blue doesn't mean the house doesn't need to be painted.

Re: For AI leaders Doom is a form of hype

#154

Post by pixl97 »

cmrdporcupine wrote: post #146
pixl97 wrote: post #138

Earlier quoted context omitted.

> They are executing loss functions and reward systems and are responding to prompts. And because we trained them on the entirety of human knowledge this happens to look like self-preservation, fear of death, and instrumental goals. I mean, we train them in an evolutionary manner. Instrumental convergence will never happen, right?

I mean, yes, they cosplay as us-like things. That's a neat trick. I'm not convinced that means what these fear-pilled engineers think it means tho?

OK, what does it mean then?

Re: For AI leaders Doom is a form of hype

#155

Post by SpicyLemonZest »

LogicFailsMe wrote: post #140
SpicyLemonZest wrote: post #133

Earlier quoted context omitted.

We understand "superhuman coding" to mean very different things, and I think it's a me problem that I can't figure out where you're coming from, because it doesn't sound like our practical experience differs. I'm going to have to think on this.

Define it. I haven't seen anything yet from a coding agent I couldn't write myself and usually write better. But I don't have to gold plate every line of code, just the 5% or so that eats 90+% of the cycles. And this is where the agents are weakest. This does get to one of the reasons I don't believe AI cures death or builds von Neumann replicators any time soon because that requires out of sample breakthroughs. And…

Is there a retrospective on AlphaFold you’d recommend? Maybe that’s the insight I’m missing, I didn’t realize it had broad identifiable gaps like that.

Re: For AI leaders Doom is a form of hype

#156

Post by cmrdporcupine »

pixl97 wrote: post #154
cmrdporcupine wrote: post #146

Earlier quoted context omitted.

I mean, yes, they cosplay as us-like things. That's a neat trick. I'm not convinced that means what these fear-pilled engineers think it means tho?

OK, what does it mean then?

Emitted text

Re: For AI leaders Doom is a form of hype

#157

Post by jackb4040 »

reaperducer wrote: post #153
jackb4040 wrote: post #121

Earlier quoted context omitted.

https://archive.ph/BzBjZ Read the article. The things the Chinese officials are worried about are: * Fake videos / disinformation * Cyberattacks on critical infrastructure * Data leaks * Advances in military sensing / targeting technology These concerns have little in common with the Yudkowskian / apocalyptic doom marketing the US hyperscalers are pushing now. The Times even says this explicitly in the second paragra…

Read the article. I did, thanks. I don't need you to tell me to do that. The common thread is that they both see AI as a threat. It's a different threat to each side, but still a threat. Just because they see different threats from AI doesn't mean that AI isn't a threat. If one person wants to paint the house red and another person wants to paint the house blue doesn't mean the house doesn't need to be painted.

"I was kind of leaning toward thinking all the talk of doom was a bunch of self-serving regulatory-capture hype"

"Just because they see different threats from AI doesn't mean that AI isn't a threat"

Why would you make it so blatantly obvious you're acting in bad faith? Do you think the people who post here are stupid?

Re: For AI leaders Doom is a form of hype

#158

Post by beepbooptheory »

atleastoptimal wrote: post #116
beepbooptheory wrote: post #94

Earlier quoted context omitted.

In your mind, what arguments about AI risk are ignored by these poems?

Many posit that ASI or rogue AI agents are impossible, a fantasy, or marketing hype. Many claimed that the entire HuggingFace incident was an invented conspiracy made up by OpenAI and HF. The most common issue I have with them is their insistence on rigid classification: "AI is a tool, it doesn't have intentions, or desires, only humans do", as if that rhetorical flourish eliminates years of alignment and capabilitie…

Ok I see. Is there a way to articulate the risk itself, the other position here in general, that isn't just a rebuttal of the deniers/skeptics? What is the positive position or stance (as in, not simply the negative position that rejects the naysayers) we are supposed to have? Is it that AI is currently real (in like the cultural trope sense) and does have intentions and desires? That they are greater than the sum of the parts in such a way they are a clear and present danger? Or is it something else? Maybe that the "risk" is it could get to that, but we aren't yet?

There is a lot of risk involved with giving some children weapons and sending them off to recess. But we also speak of the "risk" of climate change and cigarettes. Or the "risk" of illiteracy among a population. All I am kinda curious to know (from anyone) really is what kind of risk is the one we are supposed to be taking seriously, as the non-hysterical adults-in-the-room? As the scientists contra these poets? Is it more the former, or the latter?

Just even: what is the risk specifically? What is it that author ignores here?

Re: For AI leaders Doom is a form of hype

#159

Post by reaperducer »

jackb4040 wrote: post #157
reaperducer wrote: post #153

Earlier quoted context omitted.

Read the article. I did, thanks. I don't need you to tell me to do that. The common thread is that they both see AI as a threat. It's a different threat to each side, but still a threat. Just because they see different threats from AI doesn't mean that AI isn't a threat. If one person wants to paint the house red and another person wants to paint the house blue doesn't mean the house doesn't need to be painted.

"I was kind of leaning toward thinking all the talk of doom was a bunch of self-serving regulatory-capture hype" "Just because they see different threats from AI doesn't mean that AI isn't a threat" Why would you make it so blatantly obvious you're acting in bad faith? Do you think the people who post here are stupid?

I really have no idea what you're talking about.

With that level of snap hostility toward strangers online, maybe talk to your doctor about upping your meds.

Re: For AI leaders Doom is a form of hype

#160

Post by ASalazarMX »

conmod278 wrote: post #88
NoahZuniga wrote: post #19

> I do not doubt that these people are sincere in some private, psychological sense ... . As a media scholar, what interests me is not whether Hubinger believes his own number. What interests me is what this genre of utterance does in the world ... . My complaint is not that these statements are alarming. It is that they are apocalyptic in form, hypocritical in structure, and strategically productive in effect — and…

As Von Neumann said about Oppenheimer's performative angst over the atomic bomb: "Some people profess guilt to claim credit for the sin."

This is a great quote. I liked it so much that I had to look up where it is from.

"Adventures of a Mathematician", by Stanislaw M. Ulam.

> The Polish-born mathematician recalls his formative and early professional years in his pre-war homeland and his work and colleagues at Princeton, Harvard, Wisconsin, and Los Alamos.

https://archive.org/details/adventuresofmath0000ulam_v9h3/pa...

Post reply on HN