Live data from Hacker News

The illusion of AI’s existential risk

noemamag.com

61–70 of 154 posts

Re: The illusion of AI’s existential risk

#61
post #58

Earlier quoted context omitted.

>This is a book from 2006 accurately using the salient features of LLMs popularised in the 2020s I'm a huge fan of Peter Watts' work, but to be fair that is at the very least a close-cousin to the Chinese Room thought experiment that's been around for a while.

And so it is said in the book.

Yes, Blindsight is also the book that pointed me towards Metzinger’s Being No One.

Re: The illusion of AI’s existential risk

#62
post #21

Earlier quoted context omitted.

They don’t need to be consumer goods. $1000 spent on clothes is the same as $1000 spent on GPUs as far as GDP is concerned. Headless businesses swapping dollars around creates plenty of GDP with few to no people involved.

A common fallacy is thinking that transactions exist without a human at either end of it (usually due to many layers of complexity in between). You can talk all you want about companies selling to companies, etc., but at the end of the day everything on earth is owned by a human eventually. Even high-frequency trades between two hedge funds are done with capital that was supplied by Limited Partners who probably are…

You can own something on paper but end up having no control over it. So no, not only humans can own things.

Re: The illusion of AI’s existential risk

#63

I’m just re-reading Blindsight by Peter Watts, a novel that is amazingly prescient in — amongst other things — its predictions of this coming economic upheaval. Speaking of… one scene in the book has humans communicating with an alien that appears to have “learned” human speech by training a non-sentient LLM on human ship-to-ship transmissions, and then “fine tuned” it to achieve the desired communication goal withou…

Beautiful! Or at least beautifully prescient. I'm starting to think storytellers are smarter than they let on.

Re: The illusion of AI’s existential risk

#64

I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. We could end up in a world where 99.99% of people are redundant and useless cost centers for our GDP maximizing economic paradigm. In that case, you don’t need killer robots in order to push humanity towards near-extinction. The invisible hand will smite them. But don’t worry, TED Talk attendee. The obsoletariat mo…

The world median will be chinese or indian, but the american median will be white!

Re: The illusion of AI’s existential risk

#65

Earlier quoted context omitted.

Just a tinge of broken window fallacy there?

If I understand your point, the broken window fallacy is to say that a child who breaks a window has actually done something good for the economy by keeping glass makers and installers employed. Obviously this isn’t true because those people could be building new things instead of fixing the old. Well, a huge portion of our economy is in fixing the old. Why waste so much money on healthcare for humans when we could u…

There is more to life than GDP. Human population is quite high, IMO, though I'd rather its limit not be dictated by productivity.

Re: The illusion of AI’s existential risk

#66

I’m just re-reading Blindsight by Peter Watts, a novel that is amazingly prescient in — amongst other things — its predictions of this coming economic upheaval. Speaking of… one scene in the book has humans communicating with an alien that appears to have “learned” human speech by training a non-sentient LLM on human ship-to-ship transmissions, and then “fine tuned” it to achieve the desired communication goal withou…

A much more interesting set of books (I've never managed to finish Blindsight due to Watts writing) is Ian Banks' Culture series.

Re: The illusion of AI’s existential risk

#67

Arguments in the first part are really bad. I haven't read about ai safety that much, but still see it. For example, ASI may be able to automate manufacturing really quickly, maybe be able to convince humans easily to do what it needs, etc Moreover, why ai would not be competing with us for resources? Materials on earth are finite. Also, most likely scenario is not evil AGI, as it outlined in the article, but indiffe…

Is there a difference between evil and indifferent if you are on the receiving end? Humanity is largely indifferent to animal welfare and animals would qualify it as evil how we treat them.

Re: The illusion of AI’s existential risk

#68
post #53

Earlier quoted context omitted.

Well, no. The goal is to maximise profit. One way to do that is to minimise labour costs. But at the extreme, if there are then no consumers who can buy your goods there is no profit...

> The goal is to maximise profit No, it isn't. Profit is meaningless, money is inherently worthless. Capitalism exists to encourage the production of (new) goods and services. The system is set up to encourage people to take their capital and invest it for this purpose. Investments help business ideas that would otherwise never have been made, because the person with the idea couldn't afford it. In return the investo…

But, you won’t have any money if your labor is replaced by AI.

Re: The illusion of AI’s existential risk

#69
post #53

Earlier quoted context omitted.

Well, no. The goal is to maximise profit. One way to do that is to minimise labour costs. But at the extreme, if there are then no consumers who can buy your goods there is no profit...

> The goal is to maximise profit No, it isn't. Profit is meaningless, money is inherently worthless. Capitalism exists to encourage the production of (new) goods and services. The system is set up to encourage people to take their capital and invest it for this purpose. Investments help business ideas that would otherwise never have been made, because the person with the idea couldn't afford it. In return the investo…

You are talking past each other; You're discussing very different things - capitalism's terminal utility to humanity versus capitalism's direct operational mechanisms - without actually disagreeing substantively.

The goal of the investor isn't to provide goods & services, it's to get profits in whatever way is possible. He doesn't give a shit about goods and services. This tends to diverge from the interests of non-investors.

We have an investor class that is presently largely in control of our government and our society, whose interests as a part of the capitalist system are currently at a few percent remove from our interests as people who enjoy goods & services. That divergence, as well as the accumulated privilege opposed to our interests that this class derives from control of our government and our society, appears to grow over time with uninterrupted peacetime improvements in automation, and AI is promising almost unimaginably large improvements over the next decade.

The status quo socially liberal, economically neoliberal society is not sustainable even as a polite fiction in the face of that kind of shift.

Turning and turning in the widening gyre

The falcon cannot hear the falconer;

Things fall apart; the centre cannot hold;

Mere anarchy is loosed upon the world

As we no longer weep for serfdom-slavery, you don't have to weep for wage-slavery, but you do have to replace it with something else more functional if our society is not to completely collapse. And it's looking like we will be empowering our de facto leaders with all the blood and treasure in the world to stop us from replacing it with anything more functional. Every civil war in the world up to this point has relied upon human soldiers with their own interests and allegiances, faced with a brutal choice of risking their lives to kill their countrymen - people who have the option of not pulling the trigger. This is not going to be the case for much longer.

And that's all assuming we don't get paperclipped, assuming that Ayn Rand is the penultimate cosmic horror and there is nothing darker in the possibility space, no pressure on the right side of the Drake Equation. I don't know about that.

Re: The illusion of AI’s existential risk

#70

We live in a very confusing, transitory time. Right now there's a large chunk of people who are convinced that we are a few years away from extinction if we do not ration out GPUs and bomb illegal clusters of compute, while at the same time there's a large chunk that believes that a car can never navigate a single-lane one-way tunnel safely without a driver (ie, The Boring Company). Absolutely wild.

"Convinced that we are a few years from extinction," is an exaggeration of the pessimistic/alarmed end of the spectrum of opinion. All the forecasts I've seen have the period of danger spread out over at least 20 or 30 years, and no one claims to understand the topic well enough to be able to tell whether a particular model is dangerous even if given complete freedom to examine the source code and interview the creat…

> "Convinced that we are a few years from extinction," is an exaggeration of the pessimistic/alarmed end of the spectrum of opinion.

It really isn't. There are people quitting their jobs, protesting AI companies, and inducing panic attacks in children over their irrational fears:

https://www.wired.com/story/pause-ai-existential-risk/

Post reply on HN