Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For AI leaders Doom is a form of hype

erkansaka.net

Post reply on HN
Print view View original

61–70 of 157 posts

Re: For AI leaders Doom is a form of hype

#61

Post by frabcus »

goatlover wrote: post #55
atleastoptimal wrote: post #42

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits. This is an instance of work in a genre I call "AI la…

> they believe only they can be stewards of digital gods You do't see how absurd that sounds? Not only the idea that they can create gods, but that they are the only good stewards for such beings. It's also incredibly arrogant.

Yes, that's why the rest of society needs to make them pause.

Re: For AI leaders Doom is a form of hype

#62

Post by LogicFailsMe »

fooker wrote: post #11
Betelbuddy wrote: post #8

Earlier quoted context omitted.

>> real-time weight modifications You mean training? Yeah we call that training an LLM in my backyard...

We don't really train deployed models in realtime in any realistic sense nowadays. It used to be a thing when ML was pretty much about classifying things into buckets.

May I introduce you to the word of our lord the arc-agi-3 kaggle competition?

Because that's how they've hit 18% on a single RTX Pro 6000 with DIY RSI.

Re: For AI leaders Doom is a form of hype

#63

Post by throwitaway222 »

shimman wrote: post #46
palmotea wrote: post #30

Earlier quoted context omitted.

> The timing of the AI doom narrative could not come as a more suspect time....There's little hope investors could recoup the wild investments made into Anthropic and OpenAI Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.

[flagged]

Flagging for the "evangelical christians" stuff

Re: For AI leaders Doom is a form of hype

#64

Post by ACCount39 »

apsec112 wrote: post #23

All of the main characters here (Dario, Sam Altman, Demis Hassabis, Elon Musk) have been saying this for over ten years now, since before OpenAI or Anthropic even existed

This. They've been saying "AI is an extremely powerful, extremely dangerous technology" back when actual AIs were image classifiers that would struggle to tell a cupcake apart from a dog.

OpenAI was founded by people who didn't trust Google to pursue AI tech responsibly. Anthropic was founded by people who didn't trust OpenAI to pursue AI tech responsibly. The "AI is incredibly dangerous" was the zeitgeist of AI labs long before the "hype" - or anything resembling the modern AI capabilities.

The beliefs changed little. The state of technology has changed a whole lot.

We have AIs out in the wild, coordinating in groups to carry out autonomous cyberattacks unprompted. This used to be some prime sci-fi bullshit. It has happened multiple times now in reality.

There are very legitimate reasons to be concerned now. If today's AI can cause the HuggingFace incident if it cooks off, what about tomorrow's? Larger and more capable systems, larger agentic swarms backed by more compute?

Re: For AI leaders Doom is a form of hype

#66

Post by sailfast »

palmotea wrote: post #30
mrinterweb wrote: post #24

The timing of the AI doom narrative could not come as a more suspect time. Chinese labs are releasing open models that are nipping at the heals of the frontier models. Anthropic and OpenAI know these frontier class open weight models can threaten their valuations by changing inference to a commodity where the provider who can serve the best models the cheapest and fastest will win. Few are going to pay $50/million to…

> The timing of the AI doom narrative could not come as a more suspect time....There's little hope investors could recoup the wild investments made into Anthropic and OpenAI Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.

This is the correct approach. I think we can do both things - try to add some controls while also ensuring it’s not a complete giveaway to entrenched interests and investors.

The problem with that hope and approach is regulatory capture. Big money is now the only thing that apparently matters in shaping policy, and changing that will require a lot of very sweeping vote and policy changes that may not ever have the momentum to succeed.

Re: For AI leaders Doom is a form of hype

#67

Post by zer00eyz »

NoahZuniga wrote: post #19

> I do not doubt that these people are sincere in some private, psychological sense ... . As a media scholar, what interests me is not whether Hubinger believes his own number. What interests me is what this genre of utterance does in the world ... . My complaint is not that these statements are alarming. It is that they are apocalyptic in form, hypocritical in structure, and strategically productive in effect — and…

> Like if its true it can't be hype

https://www.npr.org/2016/07/26/487522807/why-the-public-perc...

Could AI kill us all. YES!!! Are the people who are building "AI" right now on that path? I would be hard pressed to say even "maybe".

Ultimately continuing to scale the way they have is getting increasingly expensive. Thats the quiet part that none of them are saying out loud...

AI could kill us all but we cant afford to get there, were maxed out and it doesn't look like even with 10x as much money we will is NOT good for sales.

Could a break through happen for recursive self improvement, or continuous learning or continuous training - sure. But that isnt as likely and right now NO ON can afford to run experiments because there isnt the capacity out there even if you have capital.

Re: For AI leaders Doom is a form of hype

#68

Post by matherial »

apsec112 wrote: post #54

The sources that this article cites have aged very badly and the author doesn't seem to have noticed this "But, just as happened with nanotech, the wind appears to be going out of the sales of “AI.” Some researchers suggest that we may be entering a new “AI Winter,” a period of decreased funding in the area, or at least an “AI Autumn,” as exuberance for the technology fades and expectations come back to earth." (writ…

The author appears to be an LLM. This seems to be a 100% AI-generated article, which makes the "as a media scholar" passage especially hilarious.

Re: For AI leaders Doom is a form of hype

#69

Post by frabcus »

LogicFailsMe wrote: post #58
frabcus wrote: post #28

Earlier quoted context omitted.

If it helps, this person from OpenAI explains at least something: "I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly." https://x.com/MajmudarAdam/status/2098881885200081234 The explanation is basically they have other dimensions (not just pretraining and inference compute) that scale, and they've got fairly convincing scaling laws. And they know they ca…

Stephen Hawking had superhuman intelligence. He never cured himself of ALS. Explain in great detail how a power-limited algorithm stuck in a datacenter takes over a world of humans with guns, missiles, and nukes. Nukes that are air-gapped with a human in the loop BTW. Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really,…

Stephen Hawking was a human.

Re: For AI leaders Doom is a form of hype

#70

Post by throwitaway222 »

LogicFailsMe wrote: post #58
frabcus wrote: post #28

Earlier quoted context omitted.

If it helps, this person from OpenAI explains at least something: "I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly." https://x.com/MajmudarAdam/status/2098881885200081234 The explanation is basically they have other dimensions (not just pretraining and inference compute) that scale, and they've got fairly convincing scaling laws. And they know they ca…

Stephen Hawking had superhuman intelligence. He never cured himself of ALS. Explain in great detail how a power-limited algorithm stuck in a datacenter takes over a world of humans with guns, missiles, and nukes. Nukes that are air-gapped with a human in the loop BTW. Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really,…

Non-stop pushing of narratives that stoke civil war?
Post reply on HN