Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For AI leaders Doom is a form of hype

erkansaka.net

Post reply on HN
Print view View original

71–80 of 172 posts

Re: For AI leaders Doom is a form of hype

#71

Post by mrob »

mlsu wrote: post #57
atleastoptimal wrote: post #42

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits. This is an instance of work in a genre I call "AI la…

If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that th…

Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).

Re: For AI leaders Doom is a form of hype

#72

Post by shimman »

throwitaway222 wrote: post #63
shimman wrote: post #46

Earlier quoted context omitted.

[flagged]

Flagging for the "evangelical christians" stuff

Why? I was raised as an evangelical christian. End of time sermons are quite common, along with telling followers that they are warriors of christ. This doesn't seem any different than what these MBA leaders are saying "AI will destroy humanity, I can protect it."

It's delusional but it's quite effective, luckily it doesn't work on the general population but you can get a dedicated group of people, and as we see with the current government you don't need big numbers to make a huge impact.

Re: For AI leaders Doom is a form of hype

#73

Post by dwohnitmok »

apsec112 wrote: post #54

The sources that this article cites have aged very badly and the author doesn't seem to have noticed this "But, just as happened with nanotech, the wind appears to be going out of the sales of “AI.” Some researchers suggest that we may be entering a new “AI Winter,” a period of decreased funding in the area, or at least an “AI Autumn,” as exuberance for the technology fades and expectations come back to earth." (writ…

Well it's because the article is AI-generated. Same with the Sacks tweet that was on the front page yesterday.

There's a deep irony that all of these "anti-doom" pieces are entirely AI generated.

Re: For AI leaders Doom is a form of hype

#74

Post by sailfast »

LogicFailsMe wrote: post #38
palmotea wrote: post #30

Earlier quoted context omitted.

> The timing of the AI doom narrative could not come as a more suspect time....There's little hope investors could recoup the wild investments made into Anthropic and OpenAI Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.

Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

Have you ever lived through a power outage longer than 48 hours?

Did you see what it did to people in your neighborhood?

That’s the path.

Re: For AI leaders Doom is a form of hype

#75

Post by throwitaway222 »

shimman wrote: post #72
throwitaway222 wrote: post #63

Earlier quoted context omitted.

Flagging for the "evangelical christians" stuff

Why? I was raised as an evangelical christian. End of time sermons are quite common, along with telling followers that they are warriors of christ. This doesn't seem any different than what these MBA leaders are saying "AI will destroy humanity, I can protect it." It's delusional but it's quite effective, luckily it doesn't work on the general population but you can get a dedicated group of people, and as we see with…

Just stay away from it - if @dang allows it, then someone else will justifiably start saying "on par with shia muslims" in the same way you wrote your text.

Re: For AI leaders Doom is a form of hype

#76

Post by palmotea »

throwitaway222 wrote: post #48
palmotea wrote: post #30

Earlier quoted context omitted.

> The timing of the AI doom narrative could not come as a more suspect time....There's little hope investors could recoup the wild investments made into Anthropic and OpenAI Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.

So if the end of the world can be stopped, but the only specific action to do so helps Anthropic/OpenAI (and it's proven) you don't pull that lever?

>> but make sure those actions are not in the interests of Anthropic and OpenAI investors.

> So if the end of the world can be stopped, but the only specific action to do so helps Anthropic/OpenAI (and it's proven) you don't pull that lever?

Your comment has a false premise. If "the end of the world can be stopped," there's no way that the only way would be one that "helps Anthropic/OpenAI['s investors]." That can be shown by a counterexample: nationalize them and leave the investors holding the bag, then take whatever action you're talking about.

Re: For AI leaders Doom is a form of hype

#77

Post by aaroninsf »

mlsu wrote: post #57
atleastoptimal wrote: post #42

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits. This is an instance of work in a genre I call "AI la…

If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that th…

This is not an argument, it's an unfounded claim based on a straw argument.

The threat is distributed; the problem is systemic. Social herd behavior is very real, and it's a probability space, not a certainty.

Similar case: climate change is without any question causing millions of deaths per year and may well kill billions, not to mention destroying ecosystems, and how may people do you know who have given up eating beef or flying...?

Re: For AI leaders Doom is a form of hype

#78

Post by frabcus »

LogicFailsMe wrote: post #58
frabcus wrote: post #28

Earlier quoted context omitted.

If it helps, this person from OpenAI explains at least something: "I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly." https://x.com/MajmudarAdam/status/2098881885200081234 The explanation is basically they have other dimensions (not just pretraining and inference compute) that scale, and they've got fairly convincing scaling laws. And they know they ca…

Stephen Hawking had superhuman intelligence. He never cured himself of ALS. Explain in great detail how a power-limited algorithm stuck in a datacenter takes over a world of humans with guns, missiles, and nukes. Nukes that are air-gapped with a human in the loop BTW. Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really,…

Hacks in to neoclouds for more compute (e.g. like Hugging Face incident), socially manipulates people to do things for them (e.g. like social media recommendation algorithms), hacks into lab's own training/monitoring/inference (e.g. as swarm did into OpenAIs eval cluster).

See "AI 2027" or "If Anyone Builds It, Everyone Dies" for some more ideas.

Re: For AI leaders Doom is a form of hype

#79

Post by Gregkion »

LogicFailsMe wrote: post #38
palmotea wrote: post #30

Earlier quoted context omitted.

> The timing of the AI doom narrative could not come as a more suspect time....There's little hope investors could recoup the wild investments made into Anthropic and OpenAI Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.

Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

Thats the problem with doom.

What might not affect you, might affect millions across the ocean.

A society no longer creating bullshit jobs because there is no reason for it anymore. You have some AI.

Right now a whole industry is transforming (again) but now filling the automation gap we were unable to do because of a shortage of labor or expertise.

Our society already has peak employmet reached, now AI takes away all the random digital jobs too. Someone building a website? Takes 5 minutes. Someone doing odd jobs on fiverr with blender or any other tool? Takes 5 minutes now.

Regression can happen very fast. The Romens were a high level culture and an empire. Gone.

Today you have more people on the planet then ever, everyone who might want to eat and who needs to work something for it.

AI already replaced people, already makes a lot of tasks A LOT cheaper and faster and human independent.

We never had a proper discussion as a society. Most countries have a social system but look at countries like the united states of america: if you are poor, you are fucked.

And AI is not the only thing which is now peaking: Physical AI / Robots are progressing fast, its clear today already that in 50 years these robots will be better than humans (50 years is very conservative).

If we have 10 more golden years due to restructuring and rebuilding things and preparing everything for agents and robots etc. what happens after these years?

If we have 10 or 20 more of these years, what do you think will happen to all the new kids (yours perhaps too?) and jobs and everything?

Riots? Famine? Mass migration? Mass death? Slums?

And don't respond with "enough people will survive this" bullshit. No one is saying that earth as a planet is not surviving. Its the same thing with climate change...

Re: For AI leaders Doom is a form of hype

#80

Post by ahf8Aithaex7Nai »

ianm218 wrote: post #25

What I don't really understand with these posts is that they don't at any point contend with the "doomers" argument. Like people get so caught in people acting hypocritical/ having mixed incentives/ or just in many cases personally disliking West Coast billionaire types and the kind of employees they hire. All of which seems fair to point out. But like the core argument - AI is getting better at core research tasks (…

Two things:

1. If you reduce the criticism to personal dislike, you’ve built yourself a nice straw man.

2. Where’s the clear evidence? I actually see more evidence that the whole thing is already stagnating and that there won’t be much more to squeeze out of it.

One more thing:

Self-driving cars, the metaverse, Hyperloop, crypto/Web3/NFTs, AGI announcements: I’m so sick and tired of this nonsense. We should just ignore the ramblings of these crackpots.

Post reply on HN