Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The illusion of AI’s existential risk

noemamag.com

Post reply on HN
Print view View original

51–60 of 154 posts

Re: The illusion of AI’s existential risk

#51

Post by ineedasername »

jiggawatts wrote: post #34

I’m just re-reading Blindsight by Peter Watts, a novel that is amazingly prescient in — amongst other things — its predictions of this coming economic upheaval. Speaking of… one scene in the book has humans communicating with an alien that appears to have “learned” human speech by training a non-sentient LLM on human ship-to-ship transmissions, and then “fine tuned” it to achieve the desired communication goal withou…

>This is a book from 2006 accurately using the salient features of LLMs popularised in the 2020s

I'm a huge fan of Peter Watts' work, but to be fair that is at the very least a close-cousin to the Chinese Room thought experiment that's been around for a while.

Re: The illusion of AI’s existential risk

#52

Post by version_five »

seiferteric wrote: post #47
version_five wrote: post #26

Earlier quoted context omitted.

"Introducing the AI Mirror Test, which very smart people keep failing" https://www.theverge.com/23604075/ai-chatbots-bing-chatgpt-i...

99% of what people do day to day is repetitive/unoriginal so it is not too surprising that this is happening, but I wonder if we are really any closer to an AI that can develop a novel and important scientific theory or mathematical result for instance? When can we expect AI to resolve the quantum gravity issue? If it can't do that, but eventually humans do, what does that say about AI I wonder?

We're no closer. Like you say, we're showing that more and more of what we do involves no actual intelligence, but it's in the margins that real intelligence exists. We're no closer to simulating (or even describing) the unique aspects intelligence than we were in the 50s. We just have Eliza with 200 billion parameters.

Re: The illusion of AI’s existential risk

#53

Post by Aerroon »

MattPalmer1086 wrote: post #33
Aerroon wrote: post #27

Earlier quoted context omitted.

> I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. Because that's the goal of our economic system! If everything could be made with no human labor requirement, then the price of these goods is going to trend towards zero. The disruption is going to cause pain, but ultimately that's what we've always been after.

Well, no. The goal is to maximise profit. One way to do that is to minimise labour costs. But at the extreme, if there are then no consumers who can buy your goods there is no profit...

>The goal is to maximise profit

No, it isn't. Profit is meaningless, money is inherently worthless.

Capitalism exists to encourage the production of (new) goods and services. The system is set up to encourage people to take their capital and invest it for this purpose.

Investments help business ideas that would otherwise never have been made, because the person with the idea couldn't afford it. In return the investor gets a slice of the potential future profits. But the goal of the system is the products the company creates.

Money is worthless. It only carries value based on what you can trade that money for. If I have $10, but a cup of coffee costs $1000, then my $10 are meaningless. But if a house cost $1 then my $10 is very valuable. And what you can trade your money for are the goods that are produced by the economic system.

Re: The illusion of AI’s existential risk

#54

Post by sgu999 »

cmdli wrote: post #42
janalsncm wrote: post #7

I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. We could end up in a world where 99.99% of people are redundant and useless cost centers for our GDP maximizing economic paradigm. In that case, you don’t need killer robots in order to push humanity towards near-extinction. The invisible hand will smite them. But don’t worry, TED Talk attendee. The obsoletariat mo…

Given that the number of people in agriculture declined from 90% to 10% over the past 200 years or so, I'm less worried about disruption like this. People are remarkably retrainable, and just because current jobs no longer exist doesn't mean we won't create new ones out of thin air (like marketing, or Youtubers, or film lighting specialists).

How much of that would be due to the fact that we increased productivity on essential goods which inevitably benefitted everyone to some extent? It may be very different when a large portion of the jobs being destroyed are mostly useless to the productivity to start with.

Re: The illusion of AI’s existential risk

#55

Post by theragra »

Arguments in the first part are really bad. I haven't read about ai safety that much, but still see it.

For example, ASI may be able to automate manufacturing really quickly, maybe be able to convince humans easily to do what it needs, etc Moreover, why ai would not be competing with us for resources? Materials on earth are finite.

Also, most likely scenario is not evil AGI, as it outlined in the article, but indifferent. And basing reasoning about AI on Darwinian evolution is barely useful.

Also, talk about this tribal people AI is ironically even worse then silicon valley narrow viewpoint: it is Californian liberal virtue signaling.

Re: The illusion of AI’s existential risk

#56

Post by ggm »

__loam wrote: post #23
ggm wrote: post #6

AGI doesn't ever have to happen for rapid deployment of AI based systems to cause harm. Google "robodebt australia" for what happens when government uses machine derived decisions to penalise the poor.

That point is basically what the article is about.

It can't be repeated enough apparently, because people still whine on about skymesh and "but IFF the robots wake up" when the real risk is what people believe machine derived "facts" are (I quote facts because they are highly conjectural inferences from models, and often diverge from fact pretty rapidly)

Re: The illusion of AI’s existential risk

#57

Post by lukifer »

Aerroon wrote: post #53
MattPalmer1086 wrote: post #33

Earlier quoted context omitted.

Well, no. The goal is to maximise profit. One way to do that is to minimise labour costs. But at the extreme, if there are then no consumers who can buy your goods there is no profit...

> The goal is to maximise profit No, it isn't. Profit is meaningless, money is inherently worthless. Capitalism exists to encourage the production of (new) goods and services. The system is set up to encourage people to take their capital and invest it for this purpose. Investments help business ideas that would otherwise never have been made, because the person with the idea couldn't afford it. In return the investo…

The more salient term isn't "capitalism", but rather "capitalization".

Most investment is done by converting anticipated future value (modulo some degree of risk), and encapsulating it into an asset price, which is relative to the opportunity cost of alternative investments, and the current discount rate.

And capitalization doesn't have a goal beyond honoring those future returns (or "embedded growth obligations"). It doesn't care whether those returns come from innovation, production, rent-seeking, or even a "greater sucker" Ponzi scheme. Investors (and their financial agents) operate almost entirely on Make Number Go Up ("buy low sell high"), and any incentives that result in win-win increases to human flourishing are entirely secondary effects.

Re: The illusion of AI’s existential risk

#58

Post by eulgro »

ineedasername wrote: post #51
jiggawatts wrote: post #34

I’m just re-reading Blindsight by Peter Watts, a novel that is amazingly prescient in — amongst other things — its predictions of this coming economic upheaval. Speaking of… one scene in the book has humans communicating with an alien that appears to have “learned” human speech by training a non-sentient LLM on human ship-to-ship transmissions, and then “fine tuned” it to achieve the desired communication goal withou…

>This is a book from 2006 accurately using the salient features of LLMs popularised in the 2020s I'm a huge fan of Peter Watts' work, but to be fair that is at the very least a close-cousin to the Chinese Room thought experiment that's been around for a while.

And so it is said in the book.

Re: The illusion of AI’s existential risk

#59

Post by sdwr »

atq2119 wrote: post #19
__loam wrote: post #10

Earlier quoted context omitted.

This is a problem with capitalism in general, not just with AI.

Reminds me of the point made by Ted Chiang that (roughly, from memory) when people express fear of technology, quite often what they really fear is capitalism and how it will use the technology. This goes back all the way to the Luddites, who weren't actually anti-technology/progress. They were opposed to how its benefits were captured and by whom.

There's at least 3 prongs there.

PvP - someone will use it against me

PvE - someone will do it better than me

and EvP - it will take something away from our human experience

None of those really capture the weight of tradition though. Loss of control is a hard pill to swallow. Feels better to keep on doing the old thing that kinda works, instead of the new unknown.

Re: The illusion of AI’s existential risk

#60

Post by lostmsu »

eulgro wrote: post #58
ineedasername wrote: post #51

Earlier quoted context omitted.

>This is a book from 2006 accurately using the salient features of LLMs popularised in the 2020s I'm a huge fan of Peter Watts' work, but to be fair that is at the very least a close-cousin to the Chinese Room thought experiment that's been around for a while.

And so it is said in the book.

And since the book itself is a work of fiction, it doesn't really tell us what will reality look like.
Post reply on HN