Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can computers become conscious?: My reply to Roger Penrose

scottaaronson.com

Post reply on HN
Print view View original

71–80 of 161 posts

Re: Can computers become conscious?: My reply to Roger Penrose

#71

Post by freshhawk »

andyh2 wrote: post #42
joe_the_user wrote: post #33

Earlier quoted context omitted.

That said, I feel that we are extremely far away from producing a computer-based consciousness Views of distance to the production of "conscious" machines are likely all over the place because there aren't really any clear ideas about what consciousness is and why it's important. * Is it one state of mind that represents another state of mind? We trivially can construct computers with states which reflect each other.…

I believe the broadest consensus defines consciousness as having a subjective experience. What's unclear, and some deem impossible, is how we can externally detect consciousness. A machine could act and respond in a manner indistinguishable from humans but still be unconscious.

So then define "subjective experience" without "it's that thing we have" (which thing?) or using the word subjective. This feels to me like replacing one undefined term with another.

Also Dennett has an excellent rebuttal to the concept of p-zombies (unconscious entities indistinguishable from conscious ones) imo.

Re: Can computers become conscious?: My reply to Roger Penrose

#72

Post by visarga »

Zikes wrote: post #34
Animats wrote: post #27

I've suggested before that AI focus on animal-level AI (operating in the real world) for a while and STFU about "consciousness". Blithering about consciousness doesn't seem to lead anywhere. Penrose is strange. He once sued a toilet-paper manufacturer for supposedly copying his Penrose non-repeating tiling scheme for the embossing pattern on their toilet paper. He lost.

I always assumed that AI consciousness would be an emergent property, not a feature we would (or even could) effectively develop towards.

> I always assumed that AI consciousness would be an emergent property

I think it's just perception, attention, memory, imagination and behavior learning. We have made progress on all these fields, so there is reason for optimism.

To make an analogy with AlphaGo - before the match, everyone was incredulous about it. After the match, we were stunned, but it was real. In just a few months a computer learned to beat the best human at Go. The same will be with reasoning and other high level skills.

> not a feature we would (or even could) effectively develop towards.

We, the people, are made of the same stuff as computers. We have no magic dust in us. There is no reason for consciousness to be unique to our particular way.

Re: Can computers become conscious?: My reply to Roger Penrose

#73

Post by dragonwriter »

andyh2 wrote: post #42
joe_the_user wrote: post #33

Earlier quoted context omitted.

That said, I feel that we are extremely far away from producing a computer-based consciousness Views of distance to the production of "conscious" machines are likely all over the place because there aren't really any clear ideas about what consciousness is and why it's important. * Is it one state of mind that represents another state of mind? We trivially can construct computers with states which reflect each other.…

I believe the broadest consensus defines consciousness as having a subjective experience. What's unclear, and some deem impossible, is how we can externally detect consciousness. A machine could act and respond in a manner indistinguishable from humans but still be unconscious.

> I believe the broadest consensus defines consciousness as having a subjective experience.

A useless definition from an empirical perspective.

> What's unclear, and some deem impossible, is how we can externally detect consciousness.

Defined that way, its impossible to objectively verify by definition.

Re: Can computers become conscious?: My reply to Roger Penrose

#74

Post by dwaltrip »

gpderetta wrote: post #47

The most interesting definition I have seen of consciousness is [1] "the mind thinking about the mind thinking", i.e. some sort of meta-level "diagnostic process". What I find interesting is that this definition supports awareness and an the existence of an 'I', but doesn't imply any sort of control or direct influence on the unconscious mind for which there seem to be experimental evidence against. Instead the resul…

The definition I use is very similar. I like to think of consciousness or sentience as the ability to model (or predict) one's self.

A pattern-matching entity that can match its own patterns.

I might disagree about the intelligence aspect. It seems to me any entity that is at least moderately interesting (has any kind of behavioral complexity) would need some level of intelligence in order to model itself to a respectable degree.

Re: Can computers become conscious?: My reply to Roger Penrose

#75

Post by mh-cx »

gpderetta wrote: post #47

The most interesting definition I have seen of consciousness is [1] "the mind thinking about the mind thinking", i.e. some sort of meta-level "diagnostic process". What I find interesting is that this definition supports awareness and an the existence of an 'I', but doesn't imply any sort of control or direct influence on the unconscious mind for which there seem to be experimental evidence against. Instead the resul…

Not really a definition, but my very personal take on what conciousness is:

Evolution has brought about creatures that are more successful (in regard to survival and reproduction), the more predictive power they have about their environment[1]. One could say, the better a species can simulate it's environment, the more likely it will continue to exist and be part of the ongoing evolution. Our brains are of course also just products of this evolutionary process and meanwhile very good at simulating their environment.

Now if we think about Plato's cave we could say, that all our experience is just an illusion. It's all just a projection of reality, that our brain creates. I'd go a step further and say: Experience or qualia is what the process of simulation of reality feels like. Or to put it in another way: Conciousness is the process of simulation of reality. The "better" the simulation or the more complete and closer to "reality" it is (whatever that means) - the higher the form of conciousness.

Taking this definition one must also concede that even animals and "lower" life forms have some degree of conciousness. It probably feels like something to be a streptococcus.

Of course all this is just blind speculation - but it somehow feels right to me.

[1] https://www.quantamagazine.org/20151119-life-is-information-...

Re: Can computers become conscious?: My reply to Roger Penrose

#76

Post by scott_s »

Animats wrote: post #28
andrewla wrote: post #22

> Look, clearly we’re machines governed by the laws of physics. We’re computers made of meat, as Marvin Minsky put it. This really seem inescapable to me. The remaining arguments all seem like smokescreens. Being able to copy computer programs? Reproducing the running state of even a moderately complex distributed system is effectively impossible, so this argument bothers me not at all. Even if there is some quantum…

"Reproducing the running state of even a moderately complex distributed system is effectively impossible, so this argument bothers me not at all." That's a feature of the Xen hypervisor.[1] You can even migrate a process from one machine to another without stopping it. [1] http://wiki.prgmr.com/mediawiki/index.php/Chapter_9:_Xen_Mig...

You linked to the migration of a single operating system kernel. That is very different from reproducing the running state of an entire distributed system. Migrating a single kernel is not easy, but straight-forward: you have access to all of the relevant pieces (in memory process state, and state on disks) in one place. With a distributed system, that is not so, since the the running state of a distributed system will include information in-flight, some of which is not virtualized.

It is possible to migrate the state of a distributed system by designing a protocol such that all of the components stop sending information around, so that you can be sure you did not miss anything. But I think that violates the spirit of the phrase "running state", since you essentially stopped the system. It's also not necessarily so that post-migration, the system will act the same.

Re: Can computers become conscious?: My reply to Roger Penrose

#77

Post by Bromskloss »

petra wrote: post #41
andrewla wrote: post #22

> Look, clearly we’re machines governed by the laws of physics. We’re computers made of meat, as Marvin Minsky put it. This really seem inescapable to me. The remaining arguments all seem like smokescreens. Being able to copy computer programs? Reproducing the running state of even a moderately complex distributed system is effectively impossible, so this argument bothers me not at all. Even if there is some quantum…

There are already some AI systems with basic forms of consciousness at [1]. I wonder to what human mental age they are equal to ? [1] http://www.scientificamerican.com/article/automaton-robots-b...

Really? They count that as consciousness?

Re: Can computers become conscious?: My reply to Roger Penrose

#78

Post by lgas »

visarga wrote: post #29
sickbeard wrote: post #8

My definition of conscious is being aware of what you are doing and influencing it. Can a computer running a file server realize it is only storing pirated movies and stop it it without being explicitly programmed to do so? A conscious computer could.

> Can a computer running a file server realize it is only storing pirated movies and stop it it without being explicitly programmed to do so? A conscious computer could. Can a person from 1975 know your hard drive was full of pirated content? Probably he knows no title from your library, so he could not assume one way or another. It's as if specific "programming" was missing from him, or people from 1975 aren't passi…

As a data point, I'm from 1975 and I can tell if you're hard drive is full of pirated software (but I don't care).

Re: Can computers become conscious?: My reply to Roger Penrose

#79

Post by kirrent »

gpderetta wrote: post #47

The most interesting definition I have seen of consciousness is [1] "the mind thinking about the mind thinking", i.e. some sort of meta-level "diagnostic process". What I find interesting is that this definition supports awareness and an the existence of an 'I', but doesn't imply any sort of control or direct influence on the unconscious mind for which there seem to be experimental evidence against. Instead the resul…

Douglass Hoffstadter wrote about consciousness like this in many of his metamagical themas columns. By analogy he mused that consciousness might arise as a higher level construct which would watch and supervise lower level functions. While he didn't really bother to develop his arguments too far the articles all had a very enjoyable Hosffstadterian writing style that made them a real pleasure to read.

Re: Can computers become conscious?: My reply to Roger Penrose

#80

Post by deegles »

scott_s wrote: post #76
Animats wrote: post #28

Earlier quoted context omitted.

"Reproducing the running state of even a moderately complex distributed system is effectively impossible, so this argument bothers me not at all." That's a feature of the Xen hypervisor.[1] You can even migrate a process from one machine to another without stopping it. [1] http://wiki.prgmr.com/mediawiki/index.php/Chapter_9:_Xen_Mig...

You linked to the migration of a single operating system kernel. That is very different from reproducing the running state of an entire distributed system . Migrating a single kernel is not easy, but straight-forward: you have access to all of the relevant pieces (in memory process state, and state on disks) in one place. With a distributed system, that is not so, since the the running state of a distributed system w…

So, basically the same reasons why I'd never step into a teleporter :)
Post reply on HN