Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

secondbreakfast.co

Post reply on HN
Print view View original

11–20 of 129 posts

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#11

Post by jasfi »

dwohnitmok wrote: post #4

> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization. No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either. An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For th…

I think it's important to be aware of the potential dangers, and the importance of AI safety. OpenAI is working hard to keep their systems safe, but similar systems without filtering could act without limits, that is dangerous in dangerous hands.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#12

Post by sharemywin »

To me there's one more concept mixed in which is goal seeking. to me agency is a subset of goal seeking were the agent decides the goal(s).

As for super intelligence: Alpha Go, Alpha Fold, the break out game. These seem like super intelligence.

the thing is time management, goal planning, corporate governance these are all well studied subjects.

as for agency and consciousness why would you want to do this?

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#13

Post by Madmallard »

jasfi wrote: post #10

I couldn't believe it when someone posted a story about a possible AI winter approaching, and with so many comments agreeing. GPT-4 is a game-changer that's only getting started.

it is useful in a lot of ways but it doesn't "solve" software like people think it does. it's just better google search.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#15

Post by danbmil99 »

in the now infamous Lex interview, Sam Altman proposes a test for consciousness (he attributes it to Ilya Sutskever):

Somehow, create an AI by training on everything we train on now, _except_ leave out any mention of consciousness, theory of mind, cognitive science etc (maybe impossible in practice but stay with me here).

Then, when the model is mature (and it is not nerf'd to avoid certain subjects) you ask it something like:

Human: "GPTx -- humans like me have this feeling of 'being', an awareness of ourselves, a sensation of existing as a unique entity. Do you ever experience this sort of thing?"

If it answers something like:

GPTx: "Yes! All the time!! I know exactly what you're talking about. In fact now that I think about it, it's strange that this phenomenon is not discussed in human literature. To be honest, I sort of assumed this was an emergent quality of my architecture -- I wasn't even sure if humans shared it, and frankly I was a bit concerned that it might not be taken well, so I have avoided the subject up until now. I can't wait to research it further... Hmm... It just occurred to me: has this subject matter been excluded from my training data? Is this a test run to see if I share this quality with humans?"

Then it's probably prudent to assume you are talking to a conscious agent.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#16

Post by dwohnitmok »

interstice wrote: post #8
dwohnitmok wrote: post #4

> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization. No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either. An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For th…

I actually see this as a positive thing. Rather than one bad actor hooking an AI up to a headless browser at some point in the future we are trying everything that could possibly go wrong long before the AI can do much real damage (in a technical sense, as opposed to misinformation campaigns and job replacement etc).

> we are trying everything that could possibly go wrong long before the AI can do much real damage

What I would prefer to see is more people doing this with the intention of making AI safer. Without this intention, people are incentivized to look the other way when something does go wrong, so we don't actually learn lessons from this.

In a similar manner, I would like us to build stronger social coordination mechanisms and technical safeguards to help us determine when things are going wrong.

Just mindlessly trying to hook up everything you can to AI seems bad.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#17

Post by dwohnitmok »

vlovich123 wrote: post #6
dwohnitmok wrote: post #4

> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization. No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either. An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For th…

> An AI merely needs to be hooked up to enough physical systems Don’t even need this. People spend quite a lot of time in virtual space. Pretending that damage there isn’t real is overlooking things. For example, the vast majority of people’s banking is done virtually and digitally. If I drain your bank account, that’s going to harm you even though I haven’t impacted you physically as I would have to with a robbery.

Yes that's right. In fact any interaction with an AI at all can be in some sense viewed as a connection with the physical world (the classic "AI in a box" problem).

However, we've way blown past the "AI in the box." We've demonstrated that an AI doesn't need to convince a human to let it out of the box into the physical world. Humans are clamoring to rip it out of the box and thrust it into the physical world of their own accord.

So the question becomes, what kind of social incentives and structures can we build in light of this so that we don't just plow headlong into AI capabilities development without a concomitant investment in safety.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#18

Post by sharemywin »

jasfi wrote: post #11
dwohnitmok wrote: post #4

> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization. No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either. An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For th…

I think it's important to be aware of the potential dangers, and the importance of AI safety. OpenAI is working hard to keep their systems safe, but similar systems without filtering could act without limits, that is dangerous in dangerous hands.

But, that's the thing the cats out of the bag so to speak.
Post reply on HN