Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

arxiv.org

Post reply on HN
Print view View original

61–70 of 87 posts

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#61

Post by randomcatuser »

I find this argument a bit weak:

for example, regarding human marginalization in states, it's just rehashing basic tropes about government (tldr, technology exacerbates the creation of surveillance states)

- "If the creation and interpretation of laws becomes far more complex, it may become much harder for humans to even interact with legislation and the legal system directly"

Well duh. That's why as soon as we notice these things, we pass laws against it. AI isn't posing the "existential risk", the way we set up our systems are. There are lots of dictators, coups, surveillance states today. And yet, there are more places in which society functions decently well.

So overall, I'm more of the opinion that people will adapt and make things better for themselves. All this anthropomorphization of "the state" and "AI" obscures the basic premise, which is we created all this stuff, and we can (and have) modified the system to suit human flourishing

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#62

Post by llamaimperative »

alephnerd wrote: post #60
llamaimperative wrote: post #59

Earlier quoted context omitted.

I also wasn’t commenting on how a model is created…? Do you disagree there are significant incentives to scale up the power, ubiquity, and direct physical impact of these systems? Please answer with yes or no, and not some snarky “bad science fiction” straw man.

While I do think there is an incentive to scale up physical infrastructure, there is a lot of "AI Washing" happening in the space, with bog standard energy projects being justified for "AI Scaling", especially because ESG as a investment category is dead, and a lot of energy investments would be bucketed under the ESG asset class. > I also wasn’t commenting on how a model is created…? That comment wasn't targeted at…

> there is a lot of "AI Washing" happening in the space

Total red herring. Your argument for why AI cannot be a risk is because it depends on so much physical infrastructure. Sure, this infrastructure isn't going to come from thin air, but if there are strong incentives to put in place then it makes no difference whether it came from thin air or VC checkbooks. It is then in place, ergo your rationale for why this cannot be a risk is invalid.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#63

Post by Jordan-117 »

This just underscores the feeling that most of the problems people have with AI are actually problems with rampant capitalism. Negative externalities, regulatory capture, the tragedy of the commons -- AI merely intensifies them.

I've heard it said that corporations are in many ways the forerunners of the singularity, able to act with superhuman effectiveness and marshall resources on a world-altering scale in pursuit of goals that don't necessarily align with societal welfare. This paper paints a disturbing picture of what it might look like if those paperclip (profit) maximizing systems become fully untethered from human concerns.

I was always a little skeptical of the SkyNet model of AI risk, thinking the danger was more in giving an AI owner class unchecked power and no need to care about the wants or needs of a disempowered labor class (see Swanwick's "Radiant Doors" for an unsettling portrayal of this). But this scenario, where capitalism takes on a mind of its own and becomes autonomous and even predatory, feels even bleaker. It reminds me of Nick Land's dark visions of the posthuman technological future (which he's weirdly eager to see, for some reason).

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#64

Post by randomcatuser »

Another thing I don't like about this paper is how it wraps real, interesting questions in the larger framework of "existential risk" (which I don't... really think exists)

For example:

> "Instead of merely(!) aligning a single, powerful AI system, we need to align one or several complex systems that are at risk of collectively drifting away from human interests. This drift can occur even while each individual AI system successfully follows the local specification of its goals"

Well yes, making systems and incentives is a hard problem. But maybe we can specify a specific instance of this, instead of "what if one day it goes rogue!"

In our society, there are already many superhuman AI systems (in the form of companies) - and somehow, they successfully contribute to our wellbeing! In fact life is amazing (even for dumb people in society, who have equal rights). And the reason is, we have categorized the ways it goes rogue (monopoly, extortion, etc) and responded adequately.

So the "extinction by industrial dehumanization" reads a lot like "extinction by cotton mills" - i mean, look on the bright side!

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#65

Post by esafak »

azeirah wrote: post #43
teg4n_ wrote: post #18

Earlier quoted context omitted.

people already do this with algorithms. IMO AI is just a continuation of that.

AI is genuinely just a rebranding of "algorithms". Like I get it, it's faster and it works by feeding data instead of feeding code... but code is data, data is code etcetc... Jim Keller said it best, about every 10 orders of magnitude in available computation the paradigm of computing shifts to a higher level of mathematical building blocks. In the beginning we had pure logic. Then came addition and subtraction, then…

Only if animals run algorithms too. AI models are stochastic and dynamic. Algorithms are usually understood to be heuristics, fixed.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#66

Post by pram »

alephnerd wrote: post #50
phkahler wrote: post #46

Earlier quoted context omitted.

>> I think the world is more likely to end due to bog standard conflicts escalating into an actual war. Not some sort of AGI/ASI going skynet WOPPER, not skynet. You saw War Games right?

Naw, was before my time. I did see Terminator though. Did Wargames age well? I tried watching a couple iconic 80s movies like Scarface but I just couldn't stand it.

It's OK, but the climax is absolutely classic.

"A strange game. The only winning move is not to play."

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#68

Post by dbtc »

phero_cnstrcts wrote: post #26
yapyap wrote: post #7

I implore everyone to watch the movie ‘Money Monster’, not only because it’s a great movie but also because I think it has a minor plot point that basically predicts how AI will be used. (small spoiler) In Money Monster it turns out the hedge fund manager who is blaming their state of the art AI bot for malfunctioning and rapidly selling off certain stock, tanking it because of that, did so out of a machine code erro…

My best guess is that at some point a “neutral“ AI will be put in charge of everything and everybody must obey for the good of society. As in one day you only have 22 energy points to spend on electricity or another day you can only eat crickets and whatnot. But it will only appear neutral - in essence somebody will control the AI and hence control the people. And people will agree to its decisions because the AI “kn…

Replace "AI" with "The Economy" and that's pretty much how it is and has been for a while. So many "rational actors" obeying ever so willingly.

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#69

Post by Centigonal »

yapyap wrote: post #7

I implore everyone to watch the movie ‘Money Monster’, not only because it’s a great movie but also because I think it has a minor plot point that basically predicts how AI will be used. (small spoiler) In Money Monster it turns out the hedge fund manager who is blaming their state of the art AI bot for malfunctioning and rapidly selling off certain stock, tanking it because of that, did so out of a machine code erro…

This is called mathwashing

https://en.wiktionary.org/wiki/mathwashing

Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks

#70

Post by cyrillite »

ADeerAppeared wrote: post #21
alephnerd wrote: post #9

While this is a well written paper, I'm not sure it's really contextualizing realistic risks that may arise from AI. It feels like a lot of "Existential AI Risk" types are divorced from the physical aspects of maintaining software - eg. your model needs hardware to compute, you need cell towers and fiber optic cables to transmit. It feels like they always anthropomorphize AI as some sort of "God". The "AI Powered Sta…

> It feels like they always anthropomorphize AI as some sort of "God". It's not like that. It is that. They're playing Pascal's Wager against an imaginary future god. The most maddening part is that the obvious problem with that has been well identified by those circles, dubbed "Pascal's Mugging", but they're still rambling on about "extinction risk" whilst disregarding the very material ongoing issues AI causes. The…

Can you explain how pascal’s mugging functions with respect to risk rather than reward?
Post reply on HN