Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For AI leaders Doom is a form of hype

erkansaka.net

Post reply on HN
Print view View original

121–130 of 172 posts

Re: For AI leaders Doom is a form of hype

#121

Post by jackb4040 »

reaperducer wrote: post #85
NoahZuniga wrote: post #19

> I do not doubt that these people are sincere in some private, psychological sense ... . As a media scholar, what interests me is not whether Hubinger believes his own number. What interests me is what this genre of utterance does in the world ... . My complaint is not that these statements are alarming. It is that they are apocalyptic in form, hypocritical in structure, and strategically productive in effect — and…

Like if its true it can't be hype I was kind of leaning toward thinking all the talk of doom was a bunch of self-serving regulatory-capture hype until this morning when the New York Times ran a long story about the Chinese military is also saying that AI is a threat. If two sides so far apart from one another in so many ways see the same thing, there must be something to it. https://www.nytimes.com/2026/09/14/world/a…

https://archive.ph/BzBjZ

Read the article. The things the Chinese officials are worried about are:

* Fake videos / disinformation

* Cyberattacks on critical infrastructure

* Data leaks

* Advances in military sensing / targeting technology

These concerns have little in common with the Yudkowskian / apocalyptic doom marketing the US hyperscalers are pushing now. The Times even says this explicitly in the second paragraph of the article:

> "The unusually blunt assessment by Chen Yixin, the head of China’s Ministry of State Security, is IN CONTRAST TO the warnings dominating the A.I. debate in the United States, which have focused on the possibility that A.I. could slip out of human control or pose a threat to humanity itself"

Re: For AI leaders Doom is a form of hype

#122

Post by throwitaway222 »

palmotea wrote: post #76
throwitaway222 wrote: post #48

Earlier quoted context omitted.

So if the end of the world can be stopped, but the only specific action to do so helps Anthropic/OpenAI (and it's proven) you don't pull that lever?

>> but make sure those actions are not in the interests of Anthropic and OpenAI investors. > So if the end of the world can be stopped, but the only specific action to do so helps Anthropic/OpenAI (and it's proven) you don't pull that lever? Your comment has a false premise. If "the end of the world can be stopped," there's no way that the only way would be one that "helps Anthropic/OpenAI['s investors]." That can be…

Yes, but I'm forcing a tautology in my logic - if in the end it is deemed the only way to stop AI from destroying the world is something that helps Anthropic or OpenAI - then this person is asking to not perform that action.

And unfortunately my tautology is actually closer to being reality than alternatives. Stopping progress even in Chinese models (a deal with Xi for example) but that specifically helps OpenAI/Anthropic.

There are very few scenarios that would not help these two companies by adding regulation. Your counterexample is 100% less likely because it's not legal for the US to nationalize them outright without taking other action first.

Re: For AI leaders Doom is a form of hype

#123

Post by ACCount39 »

carodgers wrote: post #60

There's been substantial effort put into designing and running benchmarks for these models. You know which one they don't report any more? Chess ELO. Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess. Lecun has the correct approach. Mock the…

If chess ELO is the best proxy for general intelligence, then surely, even Deep Blue was smarter than any human alive?

You can also claim that the best proxy for general intelligence is being able to multiply large numbers. In which case both humans and LLMs lose resoundingly to an old Casio.

Pick a piss poor metric - get a piss poor result.

Also, LeCun is a fool and his "world model" approach has consistently failed to yield any advantages over either the "pure imitation learning" or modern "imitation learning annealed by RL", but that's an aside.

Re: For AI leaders Doom is a form of hype

#124

Post by jackb4040 »

ahf8Aithaex7Nai wrote: post #80
ianm218 wrote: post #25

What I don't really understand with these posts is that they don't at any point contend with the "doomers" argument. Like people get so caught in people acting hypocritical/ having mixed incentives/ or just in many cases personally disliking West Coast billionaire types and the kind of employees they hire. All of which seems fair to point out. But like the core argument - AI is getting better at core research tasks (…

Two things: 1. If you reduce the criticism to personal dislike, you’ve built yourself a nice straw man. 2. Where’s the clear evidence? I actually see more evidence that the whole thing is already stagnating and that there won’t be much more to squeeze out of it. One more thing: Self-driving cars, the metaverse, Hyperloop, crypto/Web3/NFTs, AGI announcements: I’m so sick and tired of this nonsense. We should just igno…

Thank you so much for calling out the crypto comparison!

> What I don't really understand with these posts is that they don't at any point contend with the "to the moon" argument. Like people get so caught in crypto bros acting hypocritical/ having mixed incentives/ or just in many cases personally disliking crypto billionaire types and the kind of employees they hire. All of which seems fair to point out.

> But like the core argument - crypto is getting better at core financial tasks (trading, lending, payments, asset issuance), this feedback loop could lead towards greatly accelerated financial innovation of which there is already some clear evidence, if this continues crypto will be far more powerful than traditional finance and we might get to the point where everything is onchain and we're all going to make it - is basically not even engaged with.

Re: For AI leaders Doom is a form of hype

#125

Post by carodgers »

ACCount39 wrote: post #123
carodgers wrote: post #60

There's been substantial effort put into designing and running benchmarks for these models. You know which one they don't report any more? Chess ELO. Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess. Lecun has the correct approach. Mock the…

If chess ELO is the best proxy for general intelligence, then surely, even Deep Blue was smarter than any human alive? You can also claim that the best proxy for general intelligence is being able to multiply large numbers. In which case both humans and LLMs lose resoundingly to an old Casio. Pick a piss poor metric - get a piss poor result. Also, LeCun is a fool and his "world model" approach has consistently failed…

Chess ELO is not a proxy for general intelligence because many intelligent people choose to learn things other than chess. But any intelligent person can, if they choose, learn chess and play it well.

LLMs cannot do this. They still play very poorly relative to human pros, and even modern LLMs still understand the rules so poorly that they request illegal moves.

Anyone claiming that we should be afraid of a text generator that can't understand the rules of chess is being ridiculous.

Re: For AI leaders Doom is a form of hype

#126

Post by LogicFailsMe »

SpicyLemonZest wrote: post #119
LogicFailsMe wrote: post #106

Earlier quoted context omitted.

Yes, actually I would. There's a long list of people in my career path who bet on me sticking to my guns against compelling evidence to the contrary. But I know, I know, that's impossible! Anyway, after they figured out I would change my mind, then they described me as lacking conviction because you can't win, but I digress. But upfront, it's a detailed path with specific breakthroughs and a concrete plan to achieve…

I've read Gary Marcus's critique of AI 2027. How do you grapple with the fact that superhuman coding arrived on schedule, despite his skepticism that it would? Doesn't that mean that the other predictions are also more plausible than we may think?

Superhuman coding has not happened.

I work with Codex and Claude daily. They're fantastic for script coding, config issues, and simple projects. They lose the plot on bigger things to this day. Just last night, Astra decided to cheat to present the illusion of progress until I called its BS. And I was so looking forward to finally working with the AGI. Humans remain the muse, the common sense, and the manager of making these things productive. And as much as I agree with Carmack's suggestion to not become the out of touch Kung Fu Master:

https://x.com/ID_AA_Carmack/status/2098443262214230095

working with them on projects daily gives me a pretty tactical read on what they can and cannot do IMO.

That's how I reconcile it.

But let's say it had. That's just 1 of 20 events that have to happen in sequence to hit the jackpot here.

Re: For AI leaders Doom is a form of hype

#127

Post by ahf8Aithaex7Nai »

Gregkion wrote: post #102
ahf8Aithaex7Nai wrote: post #80

Earlier quoted context omitted.

Two things: 1. If you reduce the criticism to personal dislike, you’ve built yourself a nice straw man. 2. Where’s the clear evidence? I actually see more evidence that the whole thing is already stagnating and that there won’t be much more to squeeze out of it. One more thing: Self-driving cars, the metaverse, Hyperloop, crypto/Web3/NFTs, AGI announcements: I’m so sick and tired of this nonsense. We should just igno…

2. Astra just came out and Fable 5/5.1 too. These are SIGNIFICANT better in plenty of tasks than Opus in November. Try it out. Ask Opus 4.8 to do anything in 3d. Do the same with Fable or Astra. Just because you list up some type of random technology, doesn't make it an good argument. Metaverse, Hyperloop and the crypto stuff was clearly garbage from the beginning. Easy to dismiss because all of it had massive faults…

> Just because you list up some type of random technology, doesn't make it an good argument.

In what way is this random? It's a chronology of the bullshit from the past few years. Also: easy to say in hindsight. The hype was there, and many people believed it. The press, in particular, just lapped up the claims the way a Labrador Retriever gobbles up whatever you put in front of it.

> We have opened up a potential pandoras box without knowing it.

No, we haven't. We have a new type of (I would say, inferential) computation that is truly astonishing because it can convincingly imitate language and, to a certain extent, intentional action. Feel free to read "generate" or whatever instead if you don't like "imitate." It won't wake up and murder you in your bed at night. It won't take over the world. It won't wipe out humanity. It won't fly to Mars and build data centers there. It won't maximize paperclips. It probably won't even take your job away. It has no consciousness. It has no intentions.

And if you look behind the curtain, all you'll see are a few very rich, unscrupulous guys who are stoking the debate with fearmongering because it's strategically advantageous for them to do so. Don't let them take you for a fool. Use GenAI to your advantage while tokens still cost a fraction of what they're actually worth, but ignore all that doomsday nonsense.

Re: For AI leaders Doom is a form of hype

#128

Post by pixl97 »

jackb4040 wrote: post #121
reaperducer wrote: post #85

Earlier quoted context omitted.

Like if its true it can't be hype I was kind of leaning toward thinking all the talk of doom was a bunch of self-serving regulatory-capture hype until this morning when the New York Times ran a long story about the Chinese military is also saying that AI is a threat. If two sides so far apart from one another in so many ways see the same thing, there must be something to it. https://www.nytimes.com/2026/09/14/world/a…

https://archive.ph/BzBjZ Read the article. The things the Chinese officials are worried about are: * Fake videos / disinformation * Cyberattacks on critical infrastructure * Data leaks * Advances in military sensing / targeting technology These concerns have little in common with the Yudkowskian / apocalyptic doom marketing the US hyperscalers are pushing now. The Times even says this explicitly in the second paragra…

The thing is all the above are threats.

The internet being filled with more trash than ever is a threat to our sanity.

The anti-social posts mass generated by AI is a threat to our societies.

Bullshit news stories is a threat to our political processes and democracy.

Smart cameras every are a threat to our privacy and freedom.

AI enabled scams are a threat to our wallets.

AI hacking is a threat to anything connected along with infrastructure.

Military applications of AI are a threat to everyone.

Ever increasing intelligence in agentic AI will eventually be a threat to our entire species.

All are true at the same time.

Re: For AI leaders Doom is a form of hype

#129

Post by pixl97 »

bigstrat2003 wrote: post #109
mrob wrote: post #71

Earlier quoted context omitted.

Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).

> If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. That is completely irrational.

Humans aren't exactly rational

Re: For AI leaders Doom is a form of hype

#130

Post by ahf8Aithaex7Nai »

jackb4040 wrote: post #124
ahf8Aithaex7Nai wrote: post #80

Earlier quoted context omitted.

Two things: 1. If you reduce the criticism to personal dislike, you’ve built yourself a nice straw man. 2. Where’s the clear evidence? I actually see more evidence that the whole thing is already stagnating and that there won’t be much more to squeeze out of it. One more thing: Self-driving cars, the metaverse, Hyperloop, crypto/Web3/NFTs, AGI announcements: I’m so sick and tired of this nonsense. We should just igno…

Thank you so much for calling out the crypto comparison! > What I don't really understand with these posts is that they don't at any point contend with the "to the moon" argument. Like people get so caught in crypto bros acting hypocritical/ having mixed incentives/ or just in many cases personally disliking crypto billionaire types and the kind of employees they hire. All of which seems fair to point out. > But like…

Awesome!
Post reply on HN