Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

arxiv.org

Post reply on HN
Print view View original

21–30 of 87 posts

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#21

Post by ADeerAppeared »

alephnerd wrote: post #9

While this is a well written paper, I'm not sure it's really contextualizing realistic risks that may arise from AI. It feels like a lot of "Existential AI Risk" types are divorced from the physical aspects of maintaining software - eg. your model needs hardware to compute, you need cell towers and fiber optic cables to transmit. It feels like they always anthropomorphize AI as some sort of "God". The "AI Powered Sta…

> It feels like they always anthropomorphize AI as some sort of "God".

It's not like that. It is that. They're playing Pascal's Wager against an imaginary future god.

The most maddening part is that the obvious problem with that has been well identified by those circles, dubbed "Pascal's Mugging", but they're still rambling on about "extinction risk" whilst disregarding the very material ongoing issues AI causes.

They're all clowns whose opinions are to be immediately discarded.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#22

Post by alephnerd »

xpe wrote: post #15
alephnerd wrote: post #9

While this is a well written paper, I'm not sure it's really contextualizing realistic risks that may arise from AI. It feels like a lot of "Existential AI Risk" types are divorced from the physical aspects of maintaining software - eg. your model needs hardware to compute, you need cell towers and fiber optic cables to transmit. It feels like they always anthropomorphize AI as some sort of "God". The "AI Powered Sta…

A lot hangs on what “realistic” means. The world is probabilistic, but people’s use of language often butchers the nuance. One quick way to compare events is to multiply its probability times impact.

India and China almost went to war with each other in 2020.

The decision of whether or not to shell Chinese troops and start a war between two nuclear states came down to one brigadier [0]

Conventional War between two nuclear states is a MAJOR risk, and has already happened before in 1999 (even if Clinton's State Department didn't want to call it that).

These kinds of flashpoints are way more likely to cause suffering in the near future than some sort of AGI/ASI.

[0] - https://theprint.in/defence/nearing-breaking-point-gen-narav...

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#24

Post by serviceberry »

Saline9515 wrote: post #17
alephnerd wrote: post #9

While this is a well written paper, I'm not sure it's really contextualizing realistic risks that may arise from AI. It feels like a lot of "Existential AI Risk" types are divorced from the physical aspects of maintaining software - eg. your model needs hardware to compute, you need cell towers and fiber optic cables to transmit. It feels like they always anthropomorphize AI as some sort of "God". The "AI Powered Sta…

AI powered state would be a significant improvement in many places with endemic corruption. An AI has no incentive to steal.

AI has some approximation of the incentives of whoever trained it. Unless it's handed to us from above, I don't see how that would work.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#25

Post by gorbachev »

Saline9515 wrote: post #17
alephnerd wrote: post #9

While this is a well written paper, I'm not sure it's really contextualizing realistic risks that may arise from AI. It feels like a lot of "Existential AI Risk" types are divorced from the physical aspects of maintaining software - eg. your model needs hardware to compute, you need cell towers and fiber optic cables to transmit. It feels like they always anthropomorphize AI as some sort of "God". The "AI Powered Sta…

AI powered state would be a significant improvement in many places with endemic corruption. An AI has no incentive to steal.

The people who control the AI, however, do.

AI is not some runaway Skynet type of a thing. It's controlled by people, who will use it for good and bad.

Not to mention that AI is and will be owned by people who are already concentrating power in their own hands. They're not going to, voluntarily, relinquish that power.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#26

Post by phero_cnstrcts »

yapyap wrote: post #7

I implore everyone to watch the movie ‘Money Monster’, not only because it’s a great movie but also because I think it has a minor plot point that basically predicts how AI will be used. (small spoiler) In Money Monster it turns out the hedge fund manager who is blaming their state of the art AI bot for malfunctioning and rapidly selling off certain stock, tanking it because of that, did so out of a machine code erro…

My best guess is that at some point a “neutral“ AI will be put in charge of everything and everybody must obey for the good of society. As in one day you only have 22 energy points to spend on electricity or another day you can only eat crickets and whatnot. But it will only appear neutral - in essence somebody will control the AI and hence control the people. And people will agree to its decisions because the AI “knows best.”

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#27

Post by Saline9515 »

alephnerd wrote: post #20
Saline9515 wrote: post #17

Earlier quoted context omitted.

AI powered state would be a significant improvement in many places with endemic corruption. An AI has no incentive to steal.

An "AI-Driven State" is literally what econometrics is. All of the same math, models, and techniques used in AI/ML are the exact same used in Econometrics and Optimization Theory. A pure technocratic driven state leveraging econometrics or optimization theory for the sake of it has failed multiple times. For example, the failure of the USSR's planned economy, the failure of the US's Vietnam War objectives due to a hy…

Econometrics are very far away from what current LLMs are capable of.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#28

Post by TheDudeMan »

alephnerd wrote: post #13
llamaimperative wrote: post #11

Earlier quoted context omitted.

Yeah it would only be worrisome if there were significant incentives to enlist more hardware, cell towers, and fiber optic cables to create and operate increasingly powerful AI and to improve its ability to act directly on the physical world

It's not even about that. That isn't even how a model is created. And that's what I'm getting it. It's basically bad science fiction being used to create some form of a pseudo-religious experience. At least your local Mandirs, mosques, churches, synagogues, gurdawara, and other formal religions do food drives and connect a subset of your local community. What do AI Doomers do? Nothing.

They are trying to convince the world of the problem. What else can they do?

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#29

Post by modzu »

phero_cnstrcts wrote: post #26
yapyap wrote: post #7

I implore everyone to watch the movie ‘Money Monster’, not only because it’s a great movie but also because I think it has a minor plot point that basically predicts how AI will be used. (small spoiler) In Money Monster it turns out the hedge fund manager who is blaming their state of the art AI bot for malfunctioning and rapidly selling off certain stock, tanking it because of that, did so out of a machine code erro…

My best guess is that at some point a “neutral“ AI will be put in charge of everything and everybody must obey for the good of society. As in one day you only have 22 energy points to spend on electricity or another day you can only eat crickets and whatnot. But it will only appear neutral - in essence somebody will control the AI and hence control the people. And people will agree to its decisions because the AI “kn…

The Science
Post reply on HN