The illusion of AI’s existential risk • Hacker News Board

Re: The illusion of AI’s existential risk

#101

Post by radicalace » Thu, Jul 20, 2023, 10:25 AM UTC

hashstring wrote: post #12

Yes, this is a good take. The general discussion is definitely prematurely focused on some sort of end-boss fight, while AI and large scale data collection are causing serious harm and privacy concerns to real humans already. It’s only reasonable to expect that this increases in the future. I would like to see more discussion focussing on these pressing issues. Also a pause on AI development is not an option, not a s…

name a single reason why a pause is not an option. dont say why it would be hard or unlikely or inconvenient or whatever, to exclude it as a possibility means it has to be physically impossible.

of course it isnt physically impossible. AI requires large compute, large resources. you cant advance the state of the art inconspicuously. the only reason you say it isnt an option is because you dont consider AI an existential threat, really, and therefore extreme measures arent justified. but they actually are.

Re: The illusion of AI’s existential risk

#102

Post by mjburgess » Thu, Jul 20, 2023, 10:27 AM UTC

gpderetta wrote: post #98

jiggawatts wrote: post #34

I’m just re-reading Blindsight by Peter Watts, a novel that is amazingly prescient in — amongst other things — its predictions of this coming economic upheaval. Speaking of… one scene in the book has humans communicating with an alien that appears to have “learned” human speech by training a non-sentient LLM on human ship-to-ship transmissions, and then “fine tuned” it to achieve the desired communication goal withou…

> non-sentient LLM I don't think it is described as non-sentient in the book, but only as lacking consciousness. Consciousness not being required for sentience is one of the two major themes of the book (the other being that consciousness is an evolutionary dead end). Still your example is very relevant. I very much believe that ChatGPT, for example, is not conscious. The question is if it has some amount of sentienc…

Sentience without consciousness is just cartesian dualism again: society without people; forests without trees; etc.

Sentience is a power of consciousness which is a power of the body which is a power of the cells of the body.

There is no dualism other than in the imagination: one could imagine some fuzzy distant "forest" without thereby imagining trees -- so much for the imagination.

Re: The illusion of AI’s existential risk

#103

Post by [deleted] » Thu, Jul 20, 2023, 10:36 AM UTC

radicalace wrote: post #100

zirgs wrote: post #92

Earlier quoted context omitted.

It's very hard to transport humans to other planets, but a bunch of robots are already exploring Mars. I see no reason why a superintelligence would want to confine itself to Earth. Why compete with humans on Earth if there's the whole Solar system to exploit?

you completely fail to understand it. when you understand it you wont use the word compete — thats like saying the AI will have to compete with an inanimate object. there will be no competition, no struggle and no choice in the matter. if the singularity kicks off, humans will be rendered completely helpless. you clearly dont realize this because of the ridiculous notion that you harbor of humans competing with machi…

[deleted]

Re: The illusion of AI’s existential risk

#104

Post by ilaksh » Thu, Jul 20, 2023, 10:38 AM UTC

Although I disagree with the conclusion, this looks like a pretty decent article in terms of detailed thinking.

I just think that we not only need to work on all of the more immediate problems but also worry about less immediate problems that are existential risks.

Also people always say "AGI" and they mean different things but I am guessing most now mean something like a digital living human/simulation of human when they say that. Mixed in with some blurry connotation of it automatically becoming godlike in power.

That stuff is very speculative. But we don't need AI to get to that level for it to be very dangerous. Just imagine that we have an open source GPT that is something like 33% smarter than GPT-4 and less brittle. Make the output 50 times faster than human thought and suppose that we can run it very inexpensively, in a "swarm" of agents cooperating.

Then you have a type of superintelligence that does not require any really speculative AI advance -- hyperspeed reasoning -- based on the current technology.

If that is widely deployed for military and industrial decision-making then something like a computer virus could create existential risk.

Also, realize that all these systems really need to emulate some of the core functional aspects of animals, like a self-preservation instinct or desire to control resources, is the right instruction and relaxation of guardrails. So when they can reason a bit better and at hyperspeed with collaboration between them, they don't need to be alive or anything to be dangerous.

Look at the history of increases in computing efficiency. It is easy to imagine 50 or whatever times output speed increase in less than 5 years. And even though they are not human level now, GPT-4 has proven that these types of systems can have strong problem solving ability.

Re: The illusion of AI’s existential risk

#105

Post by jazzyjackson » Thu, Jul 20, 2023, 10:38 AM UTC

flangola7 wrote: post #89

cjbprime wrote: post #83

This is a truly awful article. I'm mildly positive towards AI because I think people tend to react with fear and examine the risks without equivalently examining the rewards, and I think that's happening here and the rewards seem amazing. But the arguments of risk are serious and plausible and this article is just lazily dismissive of them. > AI is not competing for resources with human beings. A superintelligence wo…

>This is likely to become literally false sometime soon if it hasn't already, but even if it doesn't, the AI doesn't have to. It just has to convince another human that the human is in love with it and it wants the human to kill a bunch of people, then scale the process. I'm stunned at the number of people who try to make this argument. The operators of Radio Télévision Libre des Mille Collines never had to leave the…

I thought "Ex Machina" portrayed this expertly, that a superintelligence with the motive to escape would develop psychopathic manipulation well before anything resembling empathy.

Re: The illusion of AI’s existential risk

#106

Post by radicalace » Thu, Jul 20, 2023, 10:39 AM UTC

saulrh wrote: post #48

One day, someone came up with the Transformer architecture for neural networks. The ideas didn't quite come out of nowhere, but their impact did - I don't think that anyone predicted the degree to which ChatGPT and its ilk are superior to LSTMs and the like. More importantly, nobody predicted the details , the way that the GPTs behave, the way they're willing to invent facts like they're spooling out plausible though…

exactly. where were all these self proclaimed experts, who are now making confident predictions about AI, before attention is all you need? literally 100% confidently predicted AI like chatGTP was hundreds or thousands of years away, or would have if asked. these people are the real bots.

and i never saw a single person on HN or anywhere else raise concerns or even appreciate the true impact GTP would have today in 2018 when GTP was released and samples available! people coundnt get it right even when it was right in front of them. how can they think for a single second theyre getting it right now?

to my credit i recognized immediately what was going on in 2018 and to my knowledge i was the only person leaving comments warning people and begging people to support a pause of research. so by the only material metric, i am more qualified to make predictions than anyone here or even the experts who all confidently predicted that what is happening now would never happen in our childrens lifetimes.

i will never forget when hinton saw the light. when he started warning of danger and using the key words that are at the heart of the issue. and i realized i had been ahead of him. just because youre an expert in one thing, even the implementation of ML or AI, doesnt make you an expert on the broader consequences of AI.

Re: The illusion of AI’s existential risk

#107

Post by jazzyjackson » Thu, Jul 20, 2023, 10:39 AM UTC

jiggawatts wrote: post #90

jhbadger wrote: post #66

Earlier quoted context omitted.

A much more interesting set of books (I've never managed to finish Blindsight due to Watts writing) is Ian Banks' Culture series.

I rate Iain M Banks as my favourite author of all time, but the AIs in his book have a very different "flavour" compared to the intelligent-but-not-sentient aliens and AIs in Blindsight. An interesting aspect of the Culture Minds is that they're described as hybrids of human-like / neural-net / organic intelligence and also direct numerical compute and storage capacity. They can both "think" and "compute", something…

Sounds like ChatGPT with the Wolfram plug-in, no?

Re: The illusion of AI’s existential risk

#108

Post by jazzyjackson » Thu, Jul 20, 2023, 10:41 AM UTC

RandomLensman wrote: post #28

TheOtherHobbes wrote: post #14

Earlier quoted context omitted.

It's a bizarre and blinkered article. The most immediate dangers are economic. We already have a hopelessly unstable economic system, with increasing swathes of the population economically disenfranchised. AI is more likely to accelerate that than prevent it. The other immediate dangers are social and political. When one person with resources can run an AI-enhanced troll farm and social media PR engine - not even rem…

AI could also be used for counter PR. Eventually, societies could shut down social media or ban use of AI content for political purposes, for example. I would also expect a huge uptick in bureaucracy from AI, so lots of new jobs will spring up.

To me, the hype around the economic consequences feels like cover for the collapse that is already in the cards. So I agree, when the house collapses, it won't be because AI took our jobs, but the politicians will have their scapegoat.

Re: The illusion of AI’s existential risk

#109

Post by jazzyjackson » Thu, Jul 20, 2023, 10:43 AM UTC

spacebanana7 wrote: post #93

janalsncm wrote: post #7

I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. We could end up in a world where 99.99% of people are redundant and useless cost centers for our GDP maximizing economic paradigm. In that case, you don’t need killer robots in order to push humanity towards near-extinction. The invisible hand will smite them. But don’t worry, TED Talk attendee. The obsoletariat mo…

There was a time when 90% of the labour force worked in agriculture. If I lived in the 1600s I'd be terribly concerned that mass unemployment would coincide with large increases in agricultural productivity. However, when the agricultural sector became more productive the 'winners' wanted bigger houses, travel, education, healthcare etc and this provided enough opportunity for the displaced workers to find jobs. Thes…

This is a common trope, but the reality is that the winners bought the farms from folks who couldn't compete. The ex-farmers did not go on to find better compensation, they just became poor. Their children perhaps found a larger salary in the information economy, but ask them how many acres they can pass on to their grandchildren.

Re: The illusion of AI’s existential risk

#110

Post by jazzyjackson » Thu, Jul 20, 2023, 10:46 AM UTC

janalsncm wrote: post #7

I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. We could end up in a world where 99.99% of people are redundant and useless cost centers for our GDP maximizing economic paradigm. In that case, you don’t need killer robots in order to push humanity towards near-extinction. The invisible hand will smite them. But don’t worry, TED Talk attendee. The obsoletariat mo…

> The obsoletariat mostly won’t be Americans. The median member of this class will probably be Chinese or Indian.

Anyone doing physical labor is going to be just as necessary in a world of artificial mind, it's those knowledge workers of the service economy that will find their skills automated.