Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The illusion of AI’s existential risk

noemamag.com

Post reply on HN
Print view View original

111–120 of 154 posts

Re: The illusion of AI’s existential risk

#111

Post by radicalace »

eterevsky wrote: post #44
somewhereoutth wrote: post #35

Probably things would become clearer if what we currently describe as 'AI' we instead term as 'Statistical Pattern Regurgitator' or similar. Any serious attempt at AI would: need to be trained on reality - like people are trained; need to overcome the cardinality barrier - digital systems can only approximate analogue systems; and need to demonstrate spontaneous language emergence driven by survival in a social envir…

Why wouldn’t you call a human brain “statistical pattern regurgitator” as well, if it comes to this? Matching learned patterns is a big part of how brains work. I don’t see why a strong AI _needs_ to be trained specifically on reality. Imagine an alien race that sees world in X-Rays and neutrinos. An alien AI researcher claims that a “true intelligences” must train on the same inputs, and all those systems trained on…

you absolutely understand. such a key point. GTP discovers deep structural truth, and characterized this with a multidimentional heat map, about the system that produced the data its trained on. that comes from Stephen wolfram. if you trained it on clouds, it would have an autistic knowledge of clouds. what GTP is doing is exactly the same thing as human intuition. its not statistically driven, this is proven mathematically, thats why we havent seen it before. but its still just patterns and concepts, no intellect or logic.

the key observations: imagine a human with intuition so good and so powerful that you could compose an essay without thinking, as a reflex. people dont understand how insanely powerful this AI would be if it had an intellect to match its intuition. its coming. people dont understand that intuition is a necessary part of an intellect, that were already half way to true AI. these LLMs are a bundle of kindling.

Re: The illusion of AI’s existential risk

#112

Post by joshuahedlund »

cjbprime wrote: post #83

This is a truly awful article. I'm mildly positive towards AI because I think people tend to react with fear and examine the risks without equivalently examining the rewards, and I think that's happening here and the rewards seem amazing. But the arguments of risk are serious and plausible and this article is just lazily dismissive of them. > AI is not competing for resources with human beings. A superintelligence wo…

> A superintelligence will be able to convince anyone it wants to do anything for it.

What is the evidence for such an extravagant claim? Is there a correlation between IQ delta and persuadability that has no degradation as the lower IQ rises? Especially given that there are plenty of people far more intelligent than the President in your example who are much less persuasive.

Re: The illusion of AI’s existential risk

#113

Post by joshuahedlund »

ilaksh wrote: post #104

Although I disagree with the conclusion, this looks like a pretty decent article in terms of detailed thinking. I just think that we not only need to work on all of the more immediate problems but also worry about less immediate problems that are existential risks. Also people always say "AGI" and they mean different things but I am guessing most now mean something like a digital living human/simulation of human when…

imo GTP-X is going to need to be much much “smarter” and consistently accurate (which may be fundamentally impossible) before it’s going to be “widely deployed for military and industrial decision-making” anywhere.

Re: The illusion of AI’s existential risk

#114

Post by didntcheck »

cjbprime wrote: post #83

This is a truly awful article. I'm mildly positive towards AI because I think people tend to react with fear and examine the risks without equivalently examining the rewards, and I think that's happening here and the rewards seem amazing. But the arguments of risk are serious and plausible and this article is just lazily dismissive of them. > AI is not competing for resources with human beings. A superintelligence wo…

> A superintelligence will be able to convince anyone it wants to do anything for it.

This "just so" claim has always struck me as quite a reach. Humans are manipulable, yes, especially by (perceived) group pressure, but there are still limits. Especially if the human can easily just mute the AI's output. It's like when people believe that a sufficiently clever hacker can get a given program to do anything. There are millions of security vulnerabilities, yes, but you're unlikely to find a RCE in the program `true`. But instead of the 80s wizard hacker trope we're going back to the older B-movie hypnotist who can take full control of any person by talking

> You are literally linking to the person who is the most prominent voice of AI existential risk, who is seriously suggesting doing exactly that.

There's a few links in the article - which person are you referring to? Being the most prominent voice calling something unsafe is not an inherently valuable qualification. The most vocal opponents of nuclear power tend to have a poor understanding of it

Re: The illusion of AI’s existential risk

#115

Post by mptest »

cristiancavalli wrote: post #31
jefftk wrote: post #30

Earlier quoted context omitted.

> I've only seen grifters, hysterics, or the ignorant talking about existential AI risk. Which category would you put Geoffrey Hinton in? https://www.utoronto.ca/news/risks-artificial-intelligence-m...

hysterics. Remember when he said we should stop training radiologists, that’s aged pretty poorly given there’s actually a need to have more: https://www.rsna.org/news/2022/may/global-radiologist-shorta... https://mindmatters.ai/2022/08/turns-out-computers-are-not-v...

So because he made one(+) bad prediction, all future predictions are inherently poor? It seems that may be a tad of a leap? Where is the line between "poor prediction" and "hysterics"?

"Deep learning will replace radiologists within 5 years" isn't as inherently crazy as it probably seems in hindsight. Sure, it didn't pan out, but I know a vet who works for an Ai company classifying scans for training and he says first hand the models have gotten a lot better. That said, he's still not worried about being replaced.

Re: The illusion of AI’s existential risk

#116

Post by gregwebs »

The foundation of this article is that there is a zero sum game for resources for AI safety. I think we should reject this notion and improve AI safety both in terms of existential and non-extensional risk. If more resources are needed and can be used effectively for dealing with safety we should instead take resources away from propelling AI forward and apply them to safety.

I see this argument against existential risk alliance – that AI needs humans to provide it with resources- this we can hold it hostage. I like to point out that power plants are increasingly since the pandemic remotely operable- that AI will increasingly be able to hold us hostage by hacking the infrastructure we need to survive. Throw in the ability to use some robots and bio weapons to keep humans from taking them back easily.

The other aspect is that AI doesn’t have to properly conclude ahead of time that such an attempt has a high probability of success. Its program just has to decide that this will help it achieve some objective.

Re: The illusion of AI’s existential risk

#117

Post by creer »

joshuahedlund wrote: post #113
ilaksh wrote: post #104

Although I disagree with the conclusion, this looks like a pretty decent article in terms of detailed thinking. I just think that we not only need to work on all of the more immediate problems but also worry about less immediate problems that are existential risks. Also people always say "AGI" and they mean different things but I am guessing most now mean something like a digital living human/simulation of human when…

imo GTP-X is going to need to be much much “smarter” and consistently accurate (which may be fundamentally impossible) before it’s going to be “widely deployed for military and industrial decision-making” anywhere.

I don't think a piece of (AI? no, software? Meh?) thinking material needs to be above average human in its capabilities, or in any way consistently correct in its thinking, or in any way all that broadly learned, or in any way accountable - to cause significant mayhem if given half the chance.

There is this image of super-human, much faster, much broader thinking in the collective. When a bog standard human, drunk and in a foreign country, can do plenty. By this measure, chatGPTs are plenty intelligent enough - just not yet connected enough to tools, a wallet and vague orders or intentions.

Re: The illusion of AI’s existential risk

#118

Post by ChatGTP »

zirgs wrote: post #92
cjbprime wrote: post #83

This is a truly awful article. I'm mildly positive towards AI because I think people tend to react with fear and examine the risks without equivalently examining the rewards, and I think that's happening here and the rewards seem amazing. But the arguments of risk are serious and plausible and this article is just lazily dismissive of them. > AI is not competing for resources with human beings. A superintelligence wo…

It's very hard to transport humans to other planets, but a bunch of robots are already exploring Mars. I see no reason why a superintelligence would want to confine itself to Earth. Why compete with humans on Earth if there's the whole Solar system to exploit?

While I like your comment and I understand your sentiment, it's not wise to rely on this type of thing to claim we're going to be ok.

I love these type of thought experiments though...

Re: The illusion of AI’s existential risk

#119

Post by ChatGTP »

qznc wrote: post #67
theragra wrote: post #55

Arguments in the first part are really bad. I haven't read about ai safety that much, but still see it. For example, ASI may be able to automate manufacturing really quickly, maybe be able to convince humans easily to do what it needs, etc Moreover, why ai would not be competing with us for resources? Materials on earth are finite. Also, most likely scenario is not evil AGI, as it outlined in the article, but indiffe…

Is there a difference between evil and indifferent if you are on the receiving end? Humanity is largely indifferent to animal welfare and animals would qualify it as evil how we treat them.

I am currently a bit of a doomer and if I had a magic wand, I'd pause AI research for the foreseeable future because I don't think it's creating a better world for children, at least from what I've seen so far however...to your point.

I think we're only like this to animals because we are animals (we forgot), and we compete for the same type of resources they need, water, protein, calories, we're not intelligent enough to get ourselves out of this situation, but if we could, we could.

If we could we'd be able to grow tasty meat in a lab, reverse climate change, restore forrest, synthesize water cheaply, live in floating cities above beautiful habitats,,,etc etc. We're just not that smart so we do dumb shit, like factory farm.

Re: The illusion of AI’s existential risk

#120

Post by mptest »

joshuahedlund wrote: post #113
ilaksh wrote: post #104

Although I disagree with the conclusion, this looks like a pretty decent article in terms of detailed thinking. I just think that we not only need to work on all of the more immediate problems but also worry about less immediate problems that are existential risks. Also people always say "AGI" and they mean different things but I am guessing most now mean something like a digital living human/simulation of human when…

imo GTP-X is going to need to be much much “smarter” and consistently accurate (which may be fundamentally impossible) before it’s going to be “widely deployed for military and industrial decision-making” anywhere.

Sure, a llm might not be general enough/trustworthy enough yet, but transformer based analysis of drone sensory data surely is already being tested/developed? I'm no engineer but I think it is fairly obvious they'd train their own models not trust an off the shelf general LLM
Post reply on HN