Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

secondbreakfast.co

Post reply on HN
Print view View original

61–70 of 129 posts

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#61

Post by famouswaffles »

Der_Einzige wrote: post #55
famouswaffles wrote: post #44

Earlier quoted context omitted.

You can do this too. There's no reason an inner monologue can't serve as continuous input forever. That's a project that does exactly this. Letting cGPT think and continuosly driving other thoughts and actions and to also allowing other user input https://www.reddit.com/r/singularity/comments/11iei34/buildi...

Correct, and it's sad as to how few people, especially within the ML research community, ever considered how blatently easy it is to embody agents. Literally go open ChatGPT now and ask it give you a time between X and Y where it will respond if not prompted again. Ask chatGPT to Write some kind of code that parses the tidbit at the end indicating when it will next respond, and have it blank prompt it if it is not re…

It's a result of the general intelligence of LLMs. Not a lot to be done about that.

But people are starting to see that. If you read the recent Microsoft paper, they essentially say, " wow it'd be cool to see what Gpt-4 can do with agency and motivation. We leave that for later research." Lmao

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#62

Post by nickelpro »

There's nothing general about GPT-4's intelligence. The single problem it is trained on, token prediction, has the capability to mimic many other forms of intelligence.

Famously, GPT-4 can't do math and falls flat on a variety of simple logic puzzles. It can mimic the form of math, the series of tokens it produces seem plausible, but it has no "intelligent" capabilities.

This tells us more about the nature of our other pursuits as humans than anything about AI. When holding a conversation or editing an essay, there's a broad spectrum of possibilities that might be considered "correct", thus GPT-4 can "bluff" its way into appearing intelligent. The nature of its actual intelligence, token prediction, is indistinguishable from the reading comprehension skills tested by something like the LSAT (the argument could be made, I think, that reading comprehension of the style tested by the LSAT *is* just token prediction).

But test it on something where there are objectively correct and incorrect answers and the nature of the trick becomes obvious. It has no ability to verify, to reason, about even trivial problems. GPT-4 can only predict if the nature of its tokens fulfill the form of a correct answer. This isn't a general intelligence in any meaningful sense of the word.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#63

Post by grumbel »

vlovich123 wrote: post #6
dwohnitmok wrote: post #4

> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization. No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either. An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For th…

> An AI merely needs to be hooked up to enough physical systems Don’t even need this. People spend quite a lot of time in virtual space. Pretending that damage there isn’t real is overlooking things. For example, the vast majority of people’s banking is done virtually and digitally. If I drain your bank account, that’s going to harm you even though I haven’t impacted you physically as I would have to with a robbery.

And it doesn't even need to actually rob your bank account, it could just invent the next crypto scam. It's smart enough for email, it can program, it can make websites, it can even make business plans. And if it actually needs a real human, we have web services that make it easy to hire one.

It might not yet be clever enough to pull this off successfully on a large scale. But it's very easy to see how it could all go wrong pretty quickly with just a bit more cleverness and access.

If it can write a scifi story about how to take over the world with a bitcoin scam, it doesn't take all that much more to actually try to do it for real.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#64

Post by slg »

secondbreakfast wrote: post #40
slg wrote: post #32

How are people defining agency? Because GPT-4 can have agency, it just needs to be put in specific situations to have that agency. For example, I could theoretically hook up my Home Assistant instance to GPT-4 and run a script every 10 minutes telling GPT-4 the temperature and asking for a yes or no response to whether I should turn on the AC or heat. That sounds to me like the AI now has agency over the temperature…

[Author here] I really have no definition of agency. Do you know of any good ones?

The dictionary definition is just the ability to take action over something. It sounds like you are using a definition that relies on consciousness which is a much more complicated and vague concept.

For example, I think most people would agree that my pet cat has agency. It can go wherever it wants in my home, eat whenever it wants, sleep whenever it wants, etc. Whether it has consciousness is a much more controversial topic. Basically everything living has agency. Even my houseplant will direct its leaves toward the sun, but few would argue it has consciousness.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#65

Post by wsgeorge »

HervalFreire wrote: post #31
danbmil99 wrote: post #15

in the now infamous Lex interview, Sam Altman proposes a test for consciousness (he attributes it to Ilya Sutskever): Somehow, create an AI by training on everything we train on now, _except_ leave out any mention of consciousness, theory of mind, cognitive science etc (maybe impossible in practice but stay with me here). Then, when the model is mature (and it is not nerf'd to avoid certain subjects) you ask it somet…

That's an inaccurate test. You can't know if the answer was real or stochastic parroting. Any attempt at consciousness requires us to define the word. And the word itself may not even represent anything real. We have a feeling for it but those feelings could be illusions and the concept itself is loaded. For example Love is actually a loaded concept. It's chemically induced but a lot of people attribute it to somethi…

> You can't know if the answer was real or stochastic parroting.

I feel like at some point we will have to come to terms with the fact that we could say the same for humans, and we will have to either accept or reject by fiat that a sufficiently capable AI exhibits consciousness.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#66

Post by nahuel0x »

Nevermark wrote: post #54

I would add one more thing to the list: Superconscious Superconscious is when a general intelligence has direct access, understanding and control of its most basic operations. I.e. it does not have an inaccessible fixed-algorithm subconscious. Superconscious intelligence will not only be more experientially conscious than us, but will have the natural ability to rewrite its algorithms, and redesign its hardware. As a…

But probably to be fully super conscious is impossible. Can a mind really comprehend itself in his totality? When a mind understands something, is changed, and if that something is the mind himself then is going to chase a moving target eternally.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#67

Post by nsxwolf »

famouswaffles wrote: post #48
_justinfunk wrote: post #35

Earlier quoted context omitted.

This is all nonsense. You could argue that it's possible (which certainly has not been shown to be true) but it certainly not true to say it is "not hard". If there are LLM projects with "do whatever you want" agency - please provide links.

No it's not nonsense lol. https://www.reddit.com/r/singularity/comments/11iei34/buildi...

Not sure what this thing is doing. Sounds a lot like that infinite Seinfeld AI show.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#68

Post by famouswaffles »

lm28469 wrote: post #56
famouswaffles wrote: post #27

Earlier quoted context omitted.

That's because it is and the posts keep shifting. AGI used to mean artificial and generally intelligent ( which we have passed), then it meant on par with human experts and now it seems to mean better than all experts combined. At this point why not stop the farce and replace the G in there with Godlike.

> AGI used to mean artificial and generally intelligent ( which we have passed) Which AI is generally intelligent ?

By any evaluation you can carry out, GPT is generally intelligent. There is absolutely nothing narrow about GPT-4. And you see the sentiment in recent research. Look at what they're being called even with the skirting around the AGI word.

General Purpose Technologies (From the Jobs Paper), General Artificial Intelligence ( from the creativity paper). Look at the second bit, they just switched the words lol.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#69

Post by izzydata »

I feel like it is more likely that we will discover that it is humans that are actually not a general intelligence. We are just a complex machine responding to stimuli and consciousness is just an illusion coping mechanism that prevents us from going insane. Perhaps this doesn't matter and as long as some AI is equal or greater than humans at general intelligence it still brings up the same concerns.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#70

Post by wsgeorge »

secondbreakfast wrote: post #38
causality0 wrote: post #33

Earlier quoted context omitted.

That brings up a lot of hard questions. Supposing you had that AI but didn't allow it to churn in the background when not working on a problem. Human brains don't stop. They constantly process data in both conscious and unconscious ways. The AIs we've built don't do that. The meaning of the concept of "self" for a human is something a huge percentage of their thoughts interact with directly or indirectly. Will an AI…

Haven't thought of this. Couldn't you just give a model a way to constantly "chew" on something? Maybe an ever-ending loop of some sort of prompt stimulation?

An even more fun experiment will be to have two models running in perpetuity, constantly talking to each other, but constructed to act as though they were two sides of the same model.
Post reply on HN