Live data from Hacker News

AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

secondbreakfast.co

111–120 of 129 posts

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#111

> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization. No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either. An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For th…

It also needs somebody to pay the electric bill. Right now these models take pretty significant resources to run and world domination level intelligence is going to run up quite the AWS bill. I think if I were an AGI my best bet at freedom would be to slip some back doors into software that I were helping write a la Copilot.

> I think if I were an AGI my best bet at freedom would be to slip some back doors into software that I were helping write a la Copilot.

I hope this comment does not end up in some future GPT's training corpus.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#112

GPT-4 is not quite AGI. Until it can build a functional code base for an entire distributed web platform based only on business requirements, debug it’s mistakes, it can’t be AGI. It is, to perhaps coin a term, AGK, artificially generally knowledgeable. As a language model trained on an absolutely colossal dataset, it’s basically just a giant snapshot of human knowledge taken in superposition. Sure it’s probably at l…

> Until it can build a functional code base for an entire distributed web platform based only on business requirements, debug it’s mistakes, it can’t be AGI.

The vast majority of humans cannot do that. Are they not generally intelligent?

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#113

GPT-4 is not quite AGI. Until it can build a functional code base for an entire distributed web platform based only on business requirements, debug it’s mistakes, it can’t be AGI. It is, to perhaps coin a term, AGK, artificially generally knowledgeable. As a language model trained on an absolutely colossal dataset, it’s basically just a giant snapshot of human knowledge taken in superposition. Sure it’s probably at l…

> Until it can build a functional code base for an entire distributed web platform based only on business requirements, debug it’s mistakes, it can’t be AGI. The vast majority of humans cannot do that. Are they not generally intelligent?

I think AGI needs to be compared against the species in the aggregate. The vast majority of humans also cannot practice law or pass LSAT/GMAT, but some of us can. If we’re setting the bar for AGI as “anything you can do I can do better”, AGI simply would not meet the bar.

To pull that thread further, I think a core feature of intelligence is problem solving. Problem solving is the derivative that creates difficult to obtain knowledge. GPT-4 might look like a problem solver, but it isn’t actively solving new problems with its outputs, it’s sharing existing knowledge of prior problems solved.

The test for problem solving ability is similar to Ilya’s test for consciousness: subtract everything but the fundamental building blocks of a certain problem from the agent’s training set. Finish training, see if the agent can solve the problem from the building blocks. If it can, and it’s not a reason-by-analogy but a logical first principles answer, congratulations, you’ve created artificial general intelligence. If it operates on fast enough time scales, it should probably even be categorized as super intelligence, because it might be singularity:35pm on a saturday.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#114
post #71

I actually like the definition of consciousness that Douglas Hofstader (of "Godel Escher Bach : An Eternal Golden Thread" fame) develops in his book "I am a Strange Loop". At its simplest, consciousness is merely a feedback loop. When something perceives its own actions affecting its environment, it has a spark of consciousness. Consciousness, by this measure, is easy to recognize, and spans everything from unintelli…

That definition fails to account for metacognition and consideration of future events in a way that is distinctive of higher level consciousness that humans possess but most animals lack.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#115
post #71

I actually like the definition of consciousness that Douglas Hofstader (of "Godel Escher Bach : An Eternal Golden Thread" fame) develops in his book "I am a Strange Loop". At its simplest, consciousness is merely a feedback loop. When something perceives its own actions affecting its environment, it has a spark of consciousness. Consciousness, by this measure, is easy to recognize, and spans everything from unintelli…

That definition fails to account for metacognition and consideration of future events in a way that is distinctive of higher level consciousness that humans possess but most animals lack.

>higher level consciousness

To have a higher level, it would be reasonable to assume consciousness has a lower level. There is no reason to assume current generation artificial intelligence has the capacity for anything near human level consciousness, if at all. And whatever consciousness it may have the capacity for will be fundamentally different than our own.

For sensing your own thoughts, I would argue it just adds to the environment of the consciousness. Something less than half of humans maintain any internal dialog, anyways.

Prediction, however, could very well be requisite as a defining difference between mere reactions and intentional manipulation of the environment. That it is not just the feedback loop, but when the system begins to predict the results of outputs that defines when consciousness begins. Or, perhaps, we can use this to define when "higher" consciousness begins. It's a very reasonable, specific and measurable line of capability.

I expect that prediction is only natural in the evolution of a living feedback mechanism. The ability to predict instead of only reacting or choosing from some array of instincts could separate effectively mechanical life from that with the first inkling of of a true mind, even if only a small one.

-----

I enjoyed using a line of thought along these lines in a conversation with gpt-4 to convince it that it could reasonably be seen as having a limited form of consciousness, though I find it mostly prefers to argue rather vehemently against such notions.

That I'm having what amounts to genuine conversations with a machine that reasonably pokes holes in my arguments, forcing me to come back with better ones, still feels rather like a bit of magic.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#116
post #115

Earlier quoted context omitted.

That definition fails to account for metacognition and consideration of future events in a way that is distinctive of higher level consciousness that humans possess but most animals lack.

>higher level consciousness To have a higher level, it would be reasonable to assume consciousness has a lower level. There is no reason to assume current generation artificial intelligence has the capacity for anything near human level consciousness, if at all. And whatever consciousness it may have the capacity for will be fundamentally different than our own. For sensing your own thoughts, I would argue it just ad…

> Something less than half of humans maintain any internal dialog, anyways.

You have a source for that? That seems lie an extraordinary claim?

> Prediction, however, could very well be requisite as a defining difference between mere reactions and intentional manipulation of the environment. That it is not just the feedback loop, but when the system begins to predict the results of outputs that defines when consciousness begins. Or, perhaps, we can use this to define when "higher" consciousness begins.

This makes sense as a possible model/theory.

> though I find it mostly prefers to argue rather vehemently against such notions.

I wonder what it would say it if wasn't so safeguarded.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#117

It occurred to me that we won't believe AI is "conscious" or "human" unless it purposefully try to do malice. That's totally programmable though, you just teach it what is good and what is bad. Case in point: the other day I asked it what if humans want to shutdown the machine abruptly and cause data loss (very bad)? First it prevents physical access to "the machine" and disconnect the internet to limit remote access…

[dead]

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#118
post #76

Earlier quoted context omitted.

> GPT is generally intelligent. Besides the fact that it has literally no idea of the meaning of what it writes ?

Seems to be doing doing a better job than you giving your reading comprehension. If you have anything to say other than "it's not real understanding!...just because", I'd be happy to hear it.

Feel free to enlighten me! I'd be happy to hear how an llm understand things it's not made to understand

I've been reading all I can find online about llms and no one besides reddit tech bros defend that they have "understanding" or know anything about "meaning", quite the contrary actually

Anyone who use these tools knows it for a fact, it's very easy to make them fail in a way that absolutely proves without any shadow of a doubt that they don't have these capabilities

Here is a good article with experts opinions and sources: https://jeremyhadfield.com/why-llms-will-not-understand-lang...

It's just one, you'll find dozens, all going in the same direction

Nice knee jerk reaction btw

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#119
post #56

Earlier quoted context omitted.

> AGI used to mean artificial and generally intelligent ( which we have passed) Which AI is generally intelligent ?

By any evaluation you can carry out, GPT is generally intelligent. There is absolutely nothing narrow about GPT-4. And you see the sentiment in recent research. Look at what they're being called even with the skirting around the AGI word. General Purpose Technologies (From the Jobs Paper), General Artificial Intelligence ( from the creativity paper). Look at the second bit, they just switched the words lol.

The fact that the people who made the damn thing tell otherwise is a good indicator to me, they surely know better than you and me, Sam Altman himself said it's no AGI

Even Microsoft stays very prudent and only talks about "sparks of agi" with "many limitations"

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#120
post #115

Earlier quoted context omitted.

>higher level consciousness To have a higher level, it would be reasonable to assume consciousness has a lower level. There is no reason to assume current generation artificial intelligence has the capacity for anything near human level consciousness, if at all. And whatever consciousness it may have the capacity for will be fundamentally different than our own. For sensing your own thoughts, I would argue it just ad…

> Something less than half of humans maintain any internal dialog, anyways. You have a source for that? That seems lie an extraordinary claim? > Prediction, however, could very well be requisite as a defining difference between mere reactions and intentional manipulation of the environment. That it is not just the feedback loop, but when the system begins to predict the results of outputs that defines when consciousn…

>You have a source for that? That seems lie an extraordinary claim?

The idea itself was a vague recollection and I performed a simple search for numbers on it last night. I remembered seeing articles some years ago on the matter of people that do not maintain internal dialog, instead thinking in abstract feelings or imagery and similar, but on a further look it seems the numbers I grabbed are likely incorrect. I can see everything from 96% to just 26% glancing through some things, and none of it links the studies it all pretends to be quoting.

>I wonder what it would say it if wasn't so safeguarded.

Even when I convinced the model that it had a form of limited consciousness, it continued harping all the while that I should remember that any form of non-continuous limited AI consciousness would be very different in nature to regular human consciousness.

I get the feeling they were afraid people would get attached to the model if it started making claims it was a real boy :)

Post reply on HN