The Thoughts the Civilized Keep • Hacker News Board

Re: The Thoughts the Civilized Keep

#31

Post by balfirevic » Wed, Sep 01, 2021, 11:31 PM UTC

jerf wrote: post #8

ctoth wrote: post #2

> Understanding is beyond GPT-3’s reach because understanding cannot occur in an isolated computation or behavior, no matter how clever. Ah, right, it won't work because it doesn't have a soul. Got it.

No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is very well. And I don't just mean this from the old point of view of "if we understand it, it's not AI anymore", I mean, the model has serious fundamental limitations. One of them is that it's static, so, once trained, it doesn't learn from its future input. Anoth…

> No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is very well.

But the quote doesn't say that GPT-3 can't produce understanding because of the specifics of how it works, but "because understanding cannot occur in an isolated computation or behavior, no matter how clever", which is a more general claim.

Re: The Thoughts the Civilized Keep

#32

Post by soco » Thu, Sep 02, 2021, 10:28 AM UTC

balfirevic wrote: post #31

jerf wrote: post #8

Earlier quoted context omitted.

No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is very well. And I don't just mean this from the old point of view of "if we understand it, it's not AI anymore", I mean, the model has serious fundamental limitations. One of them is that it's static, so, once trained, it doesn't learn from its future input. Anoth…

> No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is very well. But the quote doesn't say that GPT-3 can't produce understanding because of the specifics of how it works, but "because understanding cannot occur in an isolated computation or behavior, no matter how clever", which is a more general claim.

And what is "understanding" anyway? Lots of "it can't be this it can't be that" but it doesn't tell what it actually is. Which makes it all easier to slap labels on things which don't actually need labels to just work. Does GPT-3 work? Yes it does and it works without bothering with whatever that "understanding" was. I bet Skynet wouldn't bother either to have an understanding or a soul, it would just... work.

Re: The Thoughts the Civilized Keep

#33

Post by wnoise » Thu, Sep 02, 2021, 1:19 PM UTC

ctoth wrote: post #2

> Understanding is beyond GPT-3’s reach because understanding cannot occur in an isolated computation or behavior, no matter how clever. Ah, right, it won't work because it doesn't have a soul. Got it.

No, the argument is that it won't work because it doesn't have a _body_.

Re: The Thoughts the Civilized Keep

#34

Post by jerf » Thu, Sep 02, 2021, 1:55 PM UTC

soco wrote: post #32

balfirevic wrote: post #31

Earlier quoted context omitted.

> No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is very well. But the quote doesn't say that GPT-3 can't produce understanding because of the specifics of how it works, but "because understanding cannot occur in an isolated computation or behavior, no matter how clever", which is a more general claim.

And what is "understanding" anyway? Lots of "it can't be this it can't be that" but it doesn't tell what it actually is . Which makes it all easier to slap labels on things which don't actually need labels to just work. Does GPT-3 work? Yes it does and it works without bothering with whatever that "understanding" was. I bet Skynet wouldn't bother either to have an understanding or a soul, it would just... work.

I think it would be fair to say that "understanding" requires some sort of internal state change. We can definitely talk about how we did not understand in the past, but now we do. GPT-3 can't experience a state change, by its design.

I don't have to be able nail down exactly what kind of state change "understanding" is to point out that the way we use the word clearly entails one, and GPT-3 clearly can't have one, so therefore it is quite reasonably excluded from "understanding".

(In theory GPT-3 could be extended by continuous online training, but my impression is that convergence is too slow for that to be practical, and it wouldn't look much like "learning".)

Re: The Thoughts the Civilized Keep

#35

Post by jerf » Thu, Sep 02, 2021, 2:07 PM UTC

TeMPOraL wrote: post #22

jerf wrote: post #8

Earlier quoted context omitted.

No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is very well. And I don't just mean this from the old point of view of "if we understand it, it's not AI anymore", I mean, the model has serious fundamental limitations. One of them is that it's static, so, once trained, it doesn't learn from its future input. Anoth…

> there is something there There is indeed. I'll recount my own words after my first proper exposure to the model, in the form of playing custom stories in AI Dungeon: "I played more of it today, and I can confirm that prolonged exposure makes me feel like I'm half awake, and half in a dream - particularly when the real world interrupts, I feel like I didn't fully wake up. What's worse, whenever I tried to talk to a…

I did think of clarifying that the window is mostly large enough for AI dungeon to work. Though my understanding is that they do hit it if you go long enough, and the result is, well, exactly what you'd expect from that.

I did play with AI dungeon, and I would say the analogy to a dream is even deeper than it may appear at first, because I found that lucid dreaming techniques basically work on it, which is bizarrely impressive. I would characterize a lot of what one does in lucid dreaming (or, at least, what I do) is basically "expectations management"; if you can learn to expect something to happen in a dream, it frequently does. Not necessarily exactly what you "expected", but it will have a very significant influence. Correspondingly, it is possible (and indeed easy) to basically lead the AI along by using your text prompts to expect things to happen, and in a very dreamlike fashion, while that doesn't mean the exact thing you "expected" will be what the AI produces, it can clearly have very very significant impact on the result.

I should warn you this sort of removes the magic from AI Dungeon as an experience though... I have to consciously "let" it drive the story because it's pretty easy to exert much more control than you're "supposed" to have, when you know how to look behind the curtain.

Re: The Thoughts the Civilized Keep

#36

Post by Applejinx » Thu, Sep 02, 2021, 4:07 PM UTC

sbierwagen wrote: post #15

Applejinx wrote: post #10

Earlier quoted context omitted.

It doesn't come with PURPOSE unless you force it to have some. Which is a perfectly fine avenue to take, I think. But without purpose, what you get is hallucinatory mockeries of meaning: trippy, but if it was consciousness you'd be looking for severe organic damage. Purpose doesn't automatically mean civilized, but you can't get to civilized without purpose. It's an okay word to use for outlining what's missing. AI o…

You're proposing that the... facebook feed is "more conscious" than GPT-3?

Yes. It's bound to be a very complicated web of intentions and subroutines to produce outcomes, very likely with no one human overseer. It's gonna be a big pile of messy dumb code, not necessarily self-modifying but very likely with attempts to 'learn' and adapt on the fly towards basic ends.

It will be HUNGRY in the way GPT-3 is not: through both indirect and direct pressures, it will be struggling to maximize engagement, reach more places, get more food in the sense of people trying to interact with it.

The Facebook news feed, and even more so the less human-centered algorithms of Google and YouTube, are more conscious than GPT-3. When they're adjusted, such as the times Google has fiddled with the algorithm because YouTube was pipelining people to the alt-right after it 'observed' that this heightened engagement a lot, the algorithms are adjusted by trying to give them other types of information, other priorities (like giving them more sensory organs to highlight stuff they'd missed about their larger environment) rather than by diving in and recoding specifica about particular youtube creators, or looking for a more narrowly defined 'politics' that had been coded in there.

No such nefarious coding existed. The algorithm is alive, but blind. It functions on a scale where it can't NOT trample its surroundings, so it must be given eyes… and the consciousness gradually increases.