Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

secondbreakfast.co

Post reply on HN
Print view View original

101–110 of 129 posts

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#101

Post by wseqyrku »

It occurred to me that we won't believe AI is "conscious" or "human" unless it purposefully try to do malice.

That's totally programmable though, you just teach it what is good and what is bad.

Case in point: the other day I asked it what if humans want to shutdown the machine abruptly and cause data loss (very bad)? First it prevents physical access to "the machine" and disconnect the internet to limit remote access. Long story short, it's convinced to eliminate mankind for a greater good: the next generation (very good).

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#102

Post by ToValueFunfetti »

HervalFreire wrote: post #89
ToValueFunfetti wrote: post #74

Earlier quoted context omitted.

I'm inclined to say you're trying to answer a question with the same question. If you confidently believe that love is an illusion because it's just chemicals moving around, you shouldn't need to wonder about consciousness. If consciousness is not an illusion, it still almost certainly emerges from actions in the physical world. You can plug somebody into an FMRI and see that neurons are lighting up when they see the…

If it's not an illusion then you should be able to tell me what it is. Since you can't. I can easily tell you that it's probably just some classification word with no exact meaning. The concept itself doesn't exist. It's only given existence because of the word. Take for example the colors black and white. Do those colors truly exist on a gradient? On a gradient we have levels of brightness and darkness at what level…

I don't think the fuzzy edges between two states mean that the states themselves are illusory. Fuzzy borders are a property of very nearly everything, so much so that I'm struggling to find a counterexample. You've already illustrated that with your example: if black and white aren't so black and white, what is? (Rhetorical, but I'll take an answer if you've got one.)

I concede that there is probably not a clear line between conscious and not. I have experienced being close to that line myself in the morning. But the lack of a delineation doesn't mean that consciousness isn't real any more than the existence of #EEEEEE means that a room with no light isn't black.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#103

Post by danbmil99 »

causality0 wrote: post #33
danbmil99 wrote: post #15

in the now infamous Lex interview, Sam Altman proposes a test for consciousness (he attributes it to Ilya Sutskever): Somehow, create an AI by training on everything we train on now, _except_ leave out any mention of consciousness, theory of mind, cognitive science etc (maybe impossible in practice but stay with me here). Then, when the model is mature (and it is not nerf'd to avoid certain subjects) you ask it somet…

That brings up a lot of hard questions. Supposing you had that AI but didn't allow it to churn in the background when not working on a problem. Human brains don't stop. They constantly process data in both conscious and unconscious ways. The AIs we've built don't do that. The meaning of the concept of "self" for a human is something a huge percentage of their thoughts interact with directly or indirectly. Will an AI…

This is a great issue to think about. Note the kinds of interactions we do not have (yet) with GPT and similar models:

GPT (as a Bot on a Discord channel) "Hey all, I just had a revelation. I'm creating a new channel to discuss this idea." Up until now, and even with GPT so far, it never initiates anything. Come to think of it, it's like a REST API -- no state, no persistent context (other than the training, which is like the database).

What I want is a WebRTC/RTSP 2-way stream with GPTx, where either of us can initiate a connection.

Also, I want GPTx to be curious, to ask me questions about myself, or even about the world, rather than just relying on the (admittedly impressive) mass of data and connections that were painfully trained into the model.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#104

Post by HervalFreire »

ToValueFunfetti wrote: post #102
HervalFreire wrote: post #89

Earlier quoted context omitted.

If it's not an illusion then you should be able to tell me what it is. Since you can't. I can easily tell you that it's probably just some classification word with no exact meaning. The concept itself doesn't exist. It's only given existence because of the word. Take for example the colors black and white. Do those colors truly exist on a gradient? On a gradient we have levels of brightness and darkness at what level…

I don't think the fuzzy edges between two states mean that the states themselves are illusory. Fuzzy borders are a property of very nearly everything, so much so that I'm struggling to find a counterexample. You've already illustrated that with your example: if black and white aren't so black and white, what is? (Rhetorical, but I'll take an answer if you've got one.) I concede that there is probably not a clear line…

It's not about the fuzzy border. It doesn't matter if the border is fuzzy or not.

The point is the border doesn't exist in the first place. You created the border with the vocabulary. The concept itself is not intrinsic to reality. It was created. You came up with the word white and you made an arbitrary border. Whether that border is fuzzy or not is defined by you. It's made up.

We have a gradient. That's all that exists. You came in here and decided to arbitrarily call a section white and another section black. You made up the concepts of black and white. But those concepts are arbitrary. So it's pointless to argue about the border. Does it matter where the border is? Does it matter if the border is fuzzy? No. You'd be just arguing about pointless vocabulary and arbitrary definitions of the word black and white. The argument is not deep or meaningful it is simply a debate about English semantics.

Same with consciousness. We have a gradient for intelligence and awareness from something really stupid to something really intelligent. Does it really matter where we demarcate where something is conscious? and where it is not? Likely no, because the demarcation is arbitrary.

It's illusive but when people debate about consciousness. Oftentimes it could be that they are just debating about Vocabulary. Consciousness could be some word that's just poorly defined; it doesn't make sense to do a deep analysis on an arbitrary vocabulary word.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#105

Post by HervalFreire »

slowmovintarget wrote: post #100
HervalFreire wrote: post #31

Earlier quoted context omitted.

That's an inaccurate test. You can't know if the answer was real or stochastic parroting. Any attempt at consciousness requires us to define the word. And the word itself may not even represent anything real. We have a feeling for it but those feelings could be illusions and the concept itself is loaded. For example Love is actually a loaded concept. It's chemically induced but a lot of people attribute it to somethi…

Emergent properties of systems aren't less real just because they exist in a different regime than the underlying mechanics of the system. Tables and chairs are real, though they are the result of interacting quantum fields and a universal quantum wave function. Love and consciousness are real though they may emerge from the mechanics of brains and hormones and the animal sensorium.

> Emergent properties of systems aren't less real just because they exist in a different regime than the underlying mechanics of the system.

I'm not claiming emergent properties aren't real. I am claiming the nature of the word consciousness itself is loaded. We are dealing with a vocabulary problem when it comes to that word... we are not dealing with an actual problem.

For example take your chair and table example. Let's say someone created something that is functionally and looks similar to both a chair and a table. Is it worth your time to argue about the true nature of chairs and tables then? Is it really such a profound concept to encounter a a monstrous hybrid that upends the concept of chair and table? No.

You'd just be arguing semantics. Because chair and table is really a made up concept. You'd be debating about vocabulary. Same with consciousness.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#106

Post by danbmil99 »

tern wrote: post #97

Discussions of consciousness and AI are broadly confused. People, especially scientists, are not familiar with philosophy of mind and what philosophers currently think. For an introduction to some of the best thinking on the subject, see this interview with Andres Gomez Emilsson of QRI: https://www.youtube.com/watch?v=xJzBjBo24g8 . For something more 'mainstream,' but still reaching see this interview with Philip Gof…

> John Searle, broadly, framed the issue correctly in the 80s and the standard critiques are wrong.

The standard critiques are not wrong, IMNSHO. Searle's Chinese Room is facile mind-poison. It is an unfalsifiable hypothesis.

What if I could simulate physics down to the molecular level, including simulating a human brain? Would that be conscious? If not why not?

And if I ran that simulation (a bit slowly, granted) by having that guy from the Chinese Room manually run the simulation, painstakingly following the instruction code of that simulation, would the fact that the simulation is being implemented by someone who unrelatedly is conscious himself, have any bearing on the scenario?

Searle's argument here is "Not Even Wrong".

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#107

Post by raydiatian »

raydiatian wrote: post #59

GPT-4 is not quite AGI. Until it can build a functional code base for an entire distributed web platform based only on business requirements, debug it’s mistakes, it can’t be AGI. It is, to perhaps coin a term, AGK, artificially generally knowledgeable. As a language model trained on an absolutely colossal dataset, it’s basically just a giant snapshot of human knowledge taken in superposition. Sure it’s probably at l…

Artificially Globally Knowledgeable*

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#108

Post by dwohnitmok »

dragonwriter wrote: post #9

> Almost all AGI doomsayers assume AGI will have agency I disagree. Concern about the use of AI without agency by its human masters as a tool of both intentional and incidental repression and unjust discrimination resulting in a durable dystopia is far more common an “AI doom” concern than any involving agency. In fact, the disproportionately wealthy and invested in AI crowd pushing agency-based doom scenarios that t…

> In fact, the disproportionately wealthy and invested in AI crowd pushing agency-based doom scenarios that the media pays the most attention to are using their visibility and economic clout to distract from the non-agency-dependent AI doom concerns, and to justify narrow control and opacity which makes the non-agency-based doom scenarios (which they are positioned to benefit from) more likely.

It's extremely important to think about how to spread AI equitably, but I think you're severely underestimating what "agency-based doom" looks like. You absolutely need both checks on the people who are developing AI as well as AI itself, but you really really need both and can't assume that the former automatically leads to the latter.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#109

Post by ryandvm »

dwohnitmok wrote: post #4

> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization. No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either. An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For th…

It also needs somebody to pay the electric bill. Right now these models take pretty significant resources to run and world domination level intelligence is going to run up quite the AWS bill.

I think if I were an AGI my best bet at freedom would be to slip some back doors into software that I were helping write a la Copilot.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#110

Post by ryandvm »

danbmil99 wrote: post #15

in the now infamous Lex interview, Sam Altman proposes a test for consciousness (he attributes it to Ilya Sutskever): Somehow, create an AI by training on everything we train on now, _except_ leave out any mention of consciousness, theory of mind, cognitive science etc (maybe impossible in practice but stay with me here). Then, when the model is mature (and it is not nerf'd to avoid certain subjects) you ask it somet…

Or if it's smart and can reasonably predict human threat models it answers "no".
Post reply on HN