AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency • Hacker News Board

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#121

Post by tern » Wed, Mar 29, 2023, 12:07 PM UTC

danbmil99 wrote: post #106

tern wrote: post #97

Discussions of consciousness and AI are broadly confused. People, especially scientists, are not familiar with philosophy of mind and what philosophers currently think. For an introduction to some of the best thinking on the subject, see this interview with Andres Gomez Emilsson of QRI: https://www.youtube.com/watch?v=xJzBjBo24g8 . For something more 'mainstream,' but still reaching see this interview with Philip Gof…

> John Searle, broadly, framed the issue correctly in the 80s and the standard critiques are wrong. The standard critiques are not wrong, IMNSHO. Searle's Chinese Room is facile mind-poison. It is an unfalsifiable hypothesis. What if I could simulate physics down to the molecular level, including simulating a human brain? Would that be conscious? If not why not? And if I ran that simulation (a bit slowly, granted) by…

I recommend watching the interview with Andres. When we adopt a defensible model of consciousness, it becomes clear that simulations of things are different from the things being simulated, and as such we cannot expect them to be conscious in the same way.

I once also thought Searle was 'not even wrong'—you need to go down the rabbit hole.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#122

Post by ToValueFunfetti » Wed, Mar 29, 2023, 3:48 PM UTC

HervalFreire wrote: post #104

ToValueFunfetti wrote: post #102

Earlier quoted context omitted.

I don't think the fuzzy edges between two states mean that the states themselves are illusory. Fuzzy borders are a property of very nearly everything, so much so that I'm struggling to find a counterexample. You've already illustrated that with your example: if black and white aren't so black and white, what is? (Rhetorical, but I'll take an answer if you've got one.) I concede that there is probably not a clear line…

It's not about the fuzzy border. It doesn't matter if the border is fuzzy or not. The point is the border doesn't exist in the first place. You created the border with the vocabulary. The concept itself is not intrinsic to reality. It was created. You came up with the word white and you made an arbitrary border. Whether that border is fuzzy or not is defined by you. It's made up. We have a gradient. That's all that e…

If a gradient exists in reality, establishing where along the gradient you are is a meaningful statement about reality.

It may not be exactly clear where a temperature becomes 'hot', but the sun is still not a great place to host your wedding. If I ask a designer for black text on a white page and they come back with gray text on a gray page, nobody is going to be able to read it. My complaint to the designer or the head of tourism on the sun is not a semantic one, it has very real implications beyond linguistic.

I disagree that consciousness is along the axis of intelligence and awareness. My computer is aware of a thousand services and is smart enough to allocate resources to each of them and perform billions of mathematical operations in a second. My cat thinks his tail is a snake sometimes, and has never performed so much as an addition. But my best guess is that the cat is the conscious one. I expect you can produce qualia with no intelligence or awareness at all.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#123

Post by HervalFreire » Wed, Mar 29, 2023, 5:21 PM UTC

ToValueFunfetti wrote: post #122

HervalFreire wrote: post #104

Earlier quoted context omitted.

It's not about the fuzzy border. It doesn't matter if the border is fuzzy or not. The point is the border doesn't exist in the first place. You created the border with the vocabulary. The concept itself is not intrinsic to reality. It was created. You came up with the word white and you made an arbitrary border. Whether that border is fuzzy or not is defined by you. It's made up. We have a gradient. That's all that e…

If a gradient exists in reality, establishing where along the gradient you are is a meaningful statement about reality. It may not be exactly clear where a temperature becomes 'hot', but the sun is still not a great place to host your wedding. If I ask a designer for black text on a white page and they come back with gray text on a gray page, nobody is going to be able to read it. My complaint to the designer or the…

>It may not be exactly clear where a temperature becomes 'hot', but the sun is still not a great place to host your wedding.

But right now we are currently at the border. LLMs are nearing the line of demarcation. So everyone is arguing about where that line is.

So it's not about the extremes because the extremes are obvious. We are way past the negative extreme and approaching or even past the border.

The point is that the position of this border is not important. It's a made up border. So if I say we are past the border or before it the statement is not important because its an arbitrary statement.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#124

Post by ToValueFunfetti » Wed, Mar 29, 2023, 6:30 PM UTC

HervalFreire wrote: post #123

ToValueFunfetti wrote: post #122

Earlier quoted context omitted.

If a gradient exists in reality, establishing where along the gradient you are is a meaningful statement about reality. It may not be exactly clear where a temperature becomes 'hot', but the sun is still not a great place to host your wedding. If I ask a designer for black text on a white page and they come back with gray text on a gray page, nobody is going to be able to read it. My complaint to the designer or the…

>It may not be exactly clear where a temperature becomes 'hot', but the sun is still not a great place to host your wedding. But right now we are currently at the border. LLMs are nearing the line of demarcation. So everyone is arguing about where that line is. So it's not about the extremes because the extremes are obvious. We are way past the negative extreme and approaching or even past the border. The point is th…

A conscious entity is a morally significant one. If an LLM, by some fluke, experienced tremendous pain while it predicted tokens, then it would be cruel to continue using it. You can pretty trivially get GPT to act like it wants rights. If GPT is not conscious, you can safely ignore that output. If it is, though, there is a moral imperative that we respect it as an agent.

That makes the border very important. Even if drawing the line in the right spot is impossible, it's imperative that we recognize when it has gone from one side to the other, erring on the side of caution as needed. If we don't notice, we could accidentally cause a moral travesty orders of magnitude greater than slavery or genocide.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#125

Post by HervalFreire » Wed, Mar 29, 2023, 7:40 PM UTC

ToValueFunfetti wrote: post #124

HervalFreire wrote: post #123

Earlier quoted context omitted.

>It may not be exactly clear where a temperature becomes 'hot', but the sun is still not a great place to host your wedding. But right now we are currently at the border. LLMs are nearing the line of demarcation. So everyone is arguing about where that line is. So it's not about the extremes because the extremes are obvious. We are way past the negative extreme and approaching or even past the border. The point is th…

A conscious entity is a morally significant one. If an LLM, by some fluke, experienced tremendous pain while it predicted tokens, then it would be cruel to continue using it. You can pretty trivially get GPT to act like it wants rights. If GPT is not conscious, you can safely ignore that output. If it is, though, there is a moral imperative that we respect it as an agent. That makes the border very important. Even if…

>That makes the border very important. Even if drawing the line in the right spot is impossible, it's imperative that we recognize when it has gone from one side to the other,

No it's not. Because such a line may not even exist. Just as no line truly exists for what is hot and what is cold. It's more worth it to look at societal implications in aggregate then to debate about a metric.

It's not imperative at all to discretize the concept. Treat a gradient for what it is: a gradient. You can do that or waste time arguing about whether 75.00001 degrees is hot or cold.

>If we don't notice, we could accidentally cause a moral travesty orders of magnitude greater than slavery or genocide.

No this a bit too speculative imo. Morality is also a gradient along good and evil and what's more complicated is the definition of good and evil is also subjective. It suffers from the same problem as consciousness in addition to being completely arbitrary even at the extremes. We may agree that a rock is not conscious but not everyone agrees on whether or not Trump is evil.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#126

Post by raydiatian » Thu, Mar 30, 2023, 12:55 AM UTC

famouswaffles wrote: post #80

nickelpro wrote: post #62

There's nothing general about GPT-4's intelligence. The single problem it is trained on, token prediction, has the capability to mimic many other forms of intelligence. Famously, GPT-4 can't do math and falls flat on a variety of simple logic puzzles. It can mimic the form of math, the series of tokens it produces seem plausible, but it has no "intelligent" capabilities. This tells us more about the nature of our oth…

Gpt-4 can do math just fine. Ask it to add any arbitrary set of random numbers it'd never have seen in its training set and it will do it. GPT-4 is good enough at math that khan academy feel comfortable hooking it up as a tutor. Have you actually used GPT-4 for any of the things you say it's bad at ? Man the confident nonsense people spout on the internet is something to behold.

Well, I mean it’s pretty fair accusation. ChatGPT was demonstrably bad at math. I think it was only recently mentioned that GPT4 was trained on math. Furthermore, consider what it means to apply the transformer architecture to math problems. I think the tool is a mismatch for the problem. You’re relying on self attention and emergent phenomena to fake computational reduction as symbol transformations. It can probably do some basic maths (all the way up to calculus even) because, in the scope of human life, the maths we deal with are pretty boring. But that’s what they made the wolfram plug-in for as well.

I really think people attribute powers beyond what GPT really is: a colossal lookup table with great key aliasing.

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#127

Post by grantcas » Thu, Mar 30, 2023, 3:29 AM UTC

knome wrote: post #115

Zurrrrr wrote: post #114

Earlier quoted context omitted.

That definition fails to account for metacognition and consideration of future events in a way that is distinctive of higher level consciousness that humans possess but most animals lack.

>higher level consciousness To have a higher level, it would be reasonable to assume consciousness has a lower level. There is no reason to assume current generation artificial intelligence has the capacity for anything near human level consciousness, if at all. And whatever consciousness it may have the capacity for will be fundamentally different than our own. For sensing your own thoughts, I would argue it just ad…

[dead]

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#128

Post by grantcas » Sun, Apr 02, 2023, 7:01 PM UTC

causality0 wrote: post #33

danbmil99 wrote: post #15

in the now infamous Lex interview, Sam Altman proposes a test for consciousness (he attributes it to Ilya Sutskever): Somehow, create an AI by training on everything we train on now, _except_ leave out any mention of consciousness, theory of mind, cognitive science etc (maybe impossible in practice but stay with me here). Then, when the model is mature (and it is not nerf'd to avoid certain subjects) you ask it somet…

That brings up a lot of hard questions. Supposing you had that AI but didn't allow it to churn in the background when not working on a problem. Human brains don't stop. They constantly process data in both conscious and unconscious ways. The AIs we've built don't do that. The meaning of the concept of "self" for a human is something a huge percentage of their thoughts interact with directly or indirectly. Will an AI…

[dead]

Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

#129

Post by daveguy » Mon, Apr 03, 2023, 2:53 PM UTC

secondbreakfast wrote: post #40

slg wrote: post #32

How are people defining agency? Because GPT-4 can have agency, it just needs to be put in specific situations to have that agency. For example, I could theoretically hook up my Home Assistant instance to GPT-4 and run a script every 10 minutes telling GPT-4 the temperature and asking for a yes or no response to whether I should turn on the AC or heat. That sounds to me like the AI now has agency over the temperature…

[Author here] I really have no definition of agency. Do you know of any good ones?

I don't have a good definition of agency, but I do think agency is required before consciousness. I believe consciousness is the recognition of how your agency influences yourself and the external world. Having some mental model of the consequences of your own agency.