Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We could stumble into AI catastrophe

cold-takes.com

Post reply on HN
Print view View original

91–100 of 122 posts

Re: We could stumble into AI catastrophe

#91

Post by gremlinsinc »

DerekBickerton wrote: post #5

AI will always need humans in the loop to intervene. Remember the old IBM motto: 'A computer isn't accountable so a computer should never make a management decision'.

What if man interfaces with an ai, and creates a brain that's 500x enhanced over modern humanity, and perhaps even 'linked' to other brains? Creating some sort of hive brain? Neuralink is already testing on humans or about to, and doesn't seem a far off notion that we'll be able to google or ask chatgpt questions directly from our thoughts, eventually our inner monologues might be an actual ai talking to us, etc.

Re: We could stumble into AI catastrophe

#92

Post by durbleflorp »

I strongly suggest anyone doubtful of the reasoning behind these concerns or asking questions like "well why don't we just do x to solve this?" go check out Robert Miles' YouTube channel[1].

It's quite an approachable intro, often entertaining, doesn't take a long time to get through all the core videos, and very thoroughly answers all the "why not just do x" questions if you go through the whole set.

He also does a good job of introducing a lot of the terms used in the field if you then want to go look up papers and get more into the details.

One important point he makes is that when you're making a risk assessment you have to both consider the probability of something going wrong and the scope of the potential consequences. When the potential consequences are an existential threat you don't need a high probability to take them seriously.

I also happen to think that if you watch all the material he makes a compelling argument that the odds of something going wrong are fairly high unless we start approaching AI research (and particularly safety and ethical concerns) drastically differently than we are now.

[1] https://m.youtube.com/watch?v=pYXy-A4siMw

Re: We could stumble into AI catastrophe

#93

Post by hooande »

korroziya wrote: post #64
hn_throwaway_99 wrote: post #49

Earlier quoted context omitted.

These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education. The only reason people are willing to pay tens of thousands of dollars for college is that it is a credentialed degree that people think will open up career doors. We shouldn't beat around the bush on this fact.

>These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education I agree, but you won't have a sheet of paper proving it. It's like that infamous scene from Good Will Hunting. Question is whether you want an easy way to PROVE you're educated. Which is what a license, certificate or diploma grants you.

um. if you're just learning for learning's sake... why do you need to prove that to anyone?

Re: We could stumble into AI catastrophe

#94

Post by grantcas »

It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first.

What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.

I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.

My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461

Re: We could stumble into AI catastrophe

#95

Post by grantcas »

dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

[dead]

Re: We could stumble into AI catastrophe

#96

Post by hooande »

quadcore wrote: post #50
dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

An important point is that AI does not need to be sentient or conscious to do a lot of damage. My pet theory is that consciousness could be (not saying I know for sure this is the case but it could as well be) an effect of "survival instinct". That is: "i'll take action to not die". Now this is very easy to learn from us as we do this all day long so to speak. Most of the way people react on the internet is tinted by…

> "i'll take action to not die"

This thought is entirely the result of evolution. Beings that didn't have a survival instinct didn't live to reproduce, so they don't have descendants that we can observe.

There's no guarantee that a survival instinct is a necessary or even desirable element of conscious thought. You're just describing survivorship bias in its most literal form. cue the picture of the shot up airplane

Re: We could stumble into AI catastrophe

#97

Post by AussieWog93 »

Animats wrote: post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

>- Machines should think, people should work. That's what working in an Amazon warehouse is like today. The machines tell the humans what to do. They have to; only the computers have an overview of the process. (Yeah, Marshall Brain's "Manna", which everyone here has probably read.)

I actually run some Manna-esque systems in my own small business (most notably a system for stock picking and shipping) and this isn't something to worry about in the near-term.

It greatly, greatly reduces the amount of frustration and stress people experience in mundane jobs, greatly reduces cognitive complexity and just generally makes the work more satisfying.

Maybe there's a risk of humans losing their jobs to machines, a la chapter 2 of Manna, but to be frank it's no different to the threat of outsourcing which has existed for 50+ years.

Re: We could stumble into AI catastrophe

#98

Post by FeepingCreature »

LegionMammal978 wrote: post #84
FeepingCreature wrote: post #34

Earlier quoted context omitted.

The problem is that we actually have lots of examples of the AI "secretly" being far smarter than it acts - prompt design. Tiny details in prompts can make huge differences in task performance.

How is this at all supposed to indicate hidden intelligence? Does it not just indicate that current models are limited in that they don't associate some prompts with the desired task as well as they associate other prompts with the task?

I'm not saying it indicates deliberately hidden intelligence, I'm just saying it's precedence for models already acting with less capability than they have.

Re: We could stumble into AI catastrophe

#99

Post by chucklenorris »

quadcore wrote: post #50
dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

An important point is that AI does not need to be sentient or conscious to do a lot of damage. My pet theory is that consciousness could be (not saying I know for sure this is the case but it could as well be) an effect of "survival instinct". That is: "i'll take action to not die". Now this is very easy to learn from us as we do this all day long so to speak. Most of the way people react on the internet is tinted by…

Yeah, i find the connection between consciousness and intelligence tenuous at best. Our survival imperative, thermodynamic and game theoretical constraints give rise to it and i'd argue that every living organism has a form of consciousnes, from very primitive to, well, our level. So yeah, I believe we'll have far more intelligent ais in the future that would otherwise be inert.

Ofcourse, in our foolishness, we could create an ai with the goal of survival and replication at the expense of everything else. Or self optimisation could inadvertedly do so

Re: We could stumble into AI catastrophe

#100

Post by mikewarot »

In my opinion, we're already in the midst of an AI catastrophe. Social media algorithms that decide what content to show, along with what ads, etc. are unregulated AI that are mediating our society on a previously impossible level of intimacy.

The Stasi couldn't keep up with the level of monitoring and influence that these systems already have.

Post reply on HN