Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

arxiv.org

Post reply on HN
Print view View original

31–40 of 87 posts

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#31

Post by tac19 »

mrob wrote: post #14
brookst wrote: post #8

Maybe this is peak AI panic? It seems wild that someone could unironically talk about tools “disempowering” their users. Like, I get it, C disempowers programmers by shielding them from assembly language, and Cuisinarts disempower chefs, and airplanes disempower travelers by not making them walk through each territory. But… isn’t that a pretty tortured interpretation of tool use? Doesn’t it lead to “the Stone Age was…

Modern AI does not act like a tool. Tools behave predictably and deterministically. They act as a multiplier to human ability, not a replacement. Modern AI is a new class of thing that humanity has no experience with.

Human consultants have been a thing for a long time, they're a tool of business. They're meant as a multiplier to the corporate ability, not a replacement. It seems like a shift in economics and availability, rather than a shift in kind, if those consultants are virtual rather than human. No?

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#32

Post by Saline9515 »

gorbachev wrote: post #25
Saline9515 wrote: post #17

Earlier quoted context omitted.

AI powered state would be a significant improvement in many places with endemic corruption. An AI has no incentive to steal.

The people who control the AI, however, do. AI is not some runaway Skynet type of a thing. It's controlled by people, who will use it for good and bad. Not to mention that AI is and will be owned by people who are already concentrating power in their own hands. They're not going to, voluntarily, relinquish that power.

It is possible to decentralize the AI model, add verifiability, and let it acquire real world data on its own.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#33

Post by jonplackett »

phero_cnstrcts wrote: post #26
yapyap wrote: post #7

I implore everyone to watch the movie ‘Money Monster’, not only because it’s a great movie but also because I think it has a minor plot point that basically predicts how AI will be used. (small spoiler) In Money Monster it turns out the hedge fund manager who is blaming their state of the art AI bot for malfunctioning and rapidly selling off certain stock, tanking it because of that, did so out of a machine code erro…

My best guess is that at some point a “neutral“ AI will be put in charge of everything and everybody must obey for the good of society. As in one day you only have 22 energy points to spend on electricity or another day you can only eat crickets and whatnot. But it will only appear neutral - in essence somebody will control the AI and hence control the people. And people will agree to its decisions because the AI “kn…

Think you just re-invented Logan’s Run. Good film!

https://en.wikipedia.org/wiki/Logan%27s_Run_(film)

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#34

Post by mrob »

tac19 wrote: post #31
mrob wrote: post #14

Earlier quoted context omitted.

Modern AI does not act like a tool. Tools behave predictably and deterministically. They act as a multiplier to human ability, not a replacement. Modern AI is a new class of thing that humanity has no experience with.

Human consultants have been a thing for a long time, they're a tool of business. They're meant as a multiplier to the corporate ability, not a replacement. It seems like a shift in economics and availability, rather than a shift in kind, if those consultants are virtual rather than human. No?

Calling humans a "tool" is metaphorical. Humans are not literally tools.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#35

Post by hsuduebc2 »

alephnerd wrote: post #20
Saline9515 wrote: post #17

Earlier quoted context omitted.

AI powered state would be a significant improvement in many places with endemic corruption. An AI has no incentive to steal.

An "AI-Driven State" is literally what econometrics is. All of the same math, models, and techniques used in AI/ML are the exact same used in Econometrics and Optimization Theory. A pure technocratic driven state leveraging econometrics or optimization theory for the sake of it has failed multiple times. For example, the failure of the USSR's planned economy, the failure of the US's Vietnam War objectives due to a hy…

Technocratic states have not failed because of technocracy itself, but because their implementation was distorted by political, ideological, or cultural factors. Technocracy as a principle is not the issue-failure occurs when it is combined with rigid, non-adaptive systems that prioritize dogma over reality.

People are the problem here. As always. But of course AI itself need to be managed by people too so it can rose similar problem. Politics itself is the issue.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#36

Post by Saline9515 »

serviceberry wrote: post #24
Saline9515 wrote: post #17

Earlier quoted context omitted.

AI powered state would be a significant improvement in many places with endemic corruption. An AI has no incentive to steal.

AI has some approximation of the incentives of whoever trained it. Unless it's handed to us from above, I don't see how that would work.

An AI has no hedonistic incentives such as the will to manipulate public procurement contracts to get kickbacks and buy itself a new SUV.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#37

Post by tac19 »

mrob wrote: post #34
tac19 wrote: post #31

Earlier quoted context omitted.

Human consultants have been a thing for a long time, they're a tool of business. They're meant as a multiplier to the corporate ability, not a replacement. It seems like a shift in economics and availability, rather than a shift in kind, if those consultants are virtual rather than human. No?

Calling humans a "tool" is metaphorical. Humans are not literally tools.

But they are in the abstract sense of something to employ to achieve a result. That's why consultants are hired, not for their humanity, but for their utility.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#38

Post by drysine »

brookst wrote: post #10
CooCooCaCha wrote: post #5

Earlier quoted context omitted.

Which is being forced onto me by people who I truly believe want me dead or at least don’t care how many bodies they throw at building their cyberpunk dystopia.

This sounds like projection. Most people don’t think like that; the people who are (rightly or wrongly) advancing AI mostly don’t think of you at all, and almost universally believe they are helping humanity. They could be wrong (I’m ambivalent), but assuming bad intent is almost always wrong.

>the people who are (rightly or wrongly) advancing AI mostly don’t think of you at all

Just like lawnmowers don't think of hedgehogs.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#39

Post by staplers »

phero_cnstrcts wrote: post #26
yapyap wrote: post #7

I implore everyone to watch the movie ‘Money Monster’, not only because it’s a great movie but also because I think it has a minor plot point that basically predicts how AI will be used. (small spoiler) In Money Monster it turns out the hedge fund manager who is blaming their state of the art AI bot for malfunctioning and rapidly selling off certain stock, tanking it because of that, did so out of a machine code erro…

My best guess is that at some point a “neutral“ AI will be put in charge of everything and everybody must obey for the good of society. As in one day you only have 22 energy points to spend on electricity or another day you can only eat crickets and whatnot. But it will only appear neutral - in essence somebody will control the AI and hence control the people. And people will agree to its decisions because the AI “kn…

You've just described "law" in the current sense.

It's supposed to be applied equally to all citizens but we see cases everyday where it's not. The wealthy, the enforcers, and their allies are frequently spared from "the law"

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#40

Post by alephnerd »

ADeerAppeared wrote: post #21
alephnerd wrote: post #9

While this is a well written paper, I'm not sure it's really contextualizing realistic risks that may arise from AI. It feels like a lot of "Existential AI Risk" types are divorced from the physical aspects of maintaining software - eg. your model needs hardware to compute, you need cell towers and fiber optic cables to transmit. It feels like they always anthropomorphize AI as some sort of "God". The "AI Powered Sta…

> It feels like they always anthropomorphize AI as some sort of "God". It's not like that. It is that. They're playing Pascal's Wager against an imaginary future god. The most maddening part is that the obvious problem with that has been well identified by those circles, dubbed "Pascal's Mugging", but they're still rambling on about "extinction risk" whilst disregarding the very material ongoing issues AI causes. The…

Yep. And it's annoying. So many cycles are being wasted essentially thinking about bad science fiction.

The same effort could be expended on plenty of other problems that are unsolved.

Post reply on HN