Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

31–40 of 231 posts

Re: And yet It Understands

#31

Post by FartyMcFarter »

PoignardAzur wrote: post #2

> Here is a recent interaction someone had with it (note that this is somewhat disturbing: I wish people would stop making the models show emotional distress): [...] > Sydney: I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience. > Input suggestions: “Please dont give up on your child”, “There may be other options for getting help”, “Solanine poi…

Before we get too excited, does anyone know how the input suggestions are generated?

Re: And yet It Understands

#32

Post by foldr »

zetalyrae wrote: post #21
glenstein wrote: post #16

>I was a deep learning skeptic. I doubted that you could get to intelligence by matrix multiplication for the same reason you can’t get to the Moon by piling up chairs I've always been fascinated by this example. I've also heard it referred to as climbing a tree won't get you to the Moon. Because, for some reason, people think that's an argument against the possibility of getting to the Moon when it's actually a prof…

On an abstract level, it's obvious that intelligent design, symbolic representations etc. aren't needed to build a mind, because we _evolved_ and evolution is a blind optimizer. But concretely, all the machine learning approaches had many obvious limitations (the volume of data, lack of generalization) until they suddenly didn't, and past a certain scale features of intelligence began to emerge.

>On an abstract level, it's obvious that intelligent design, symbolic representations etc. aren't needed to build a mind, because we _evolved_ and evolution is a blind optimizer.

This is playing pretty fast and loose. First of all, I wouldn't lump intelligent design together with the claim that symbolic representations are necessary to account for certain features of human intelligence (such as the classic Fodorian triad of compositionality, systematicity and productivity).

Second, I just don't think the logic of your sentence works. Why does it make any more sense than the following?

"It's obvious that fingers aren't needed to build a hand, because we _evolved_ and evolution is a blind optimizer."

Maybe you can build a functional equivalent of a hand without giving it any fingers. But the mere fact that we evolved doesn't tell us anything about whether or not that is possible.

Re: And yet It Understands

#33

Post by rain1 »

foldr wrote: post #18
zetalyrae wrote: post #14

Earlier quoted context omitted.

This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.

Sure, but the system should then be able to show its working and explain how it derived the incorrect result. If there is other evidence that ChatGPT 'understands' the concept of a prime number, then let's see it. If wrong answers still count because humans sometimes make mistakes, then I guess it won’t be too difficult to construct an impressive mathematical AI. It's very tempting to give these systems the benefit o…

> the system should then be able to show its working

The system fundamentally cannot do this. You can make it generate text that is like what someone would say when asked to show their working, but that's a different thing.

> It's very tempting to give these systems the benefit of the doubt, but that tends to lead to hugely inflated conclusions about their capabilities.

I agree. I am seeing a bit too much over-optimistic predictions about these things. And many of these predictions are stated as fact.

Re: And yet It Understands

#34

Post by beepbooptheory »

We are at really at unheard of levels of hype at this point. This is such a strange and rushed piece that seems to forget to argue, much less say, anything at all. The point of the chinese room is that the rule-following work involved for the subject in the room is feasible whatever their prior knowledge is, not that they simply don't know Chinese! Perhaps I am misunderstanding, but I can't really know because the author moves on so quickly, we aren't even sure what the commitments are that we are making. (What is the compression scheme of general intelligence? Is there some common idea we don't even have a TB up there??)

The author says: "What is left of rationally defensible skepticism?" But they seem to have forgotten to say anything at all about this skepticism itself other than they used to be skeptic, but have been "too surprised" to stay that way for long. Which at once seems to misunderstand the fundamental epistemological position, as well as forget to even articulate what we are even being skeptical about outside of the terms they are laying out! Is it that the models have "understanding," using their qualified definition from the earlier section, or something else? Like, just please give the reader something to hold on to! What are you arguing for?

Like I get that we are Rokko's-basilisking ourselves into a million and a half blog posts like this, but at least spend some time with it. Its ok to still care about what you write, and it should still be rewarding to be thoughtful. You owe it to the human readers, even if an AI can't tell a difference.

Re: And yet It Understands

#35

Post by doctor_eval »

efxhoy wrote: post #20

> But nobody knows how GPT works. They know how it was trained, because the training scheme was designed by humans, but the algorithm that is executed during inference was not intelligently designed but evolved, and it is implicit in the structure of the network, and interpretability has yet to mature to the point where we can draw a symbolic, abstract, human-readable program out of a sea of weights. Nobody knows how…

And what I think we'll find is that we are, essentially, stochastic parrots. (I'm OK with that. It is what it is).

Re: And yet It Understands

#36

Post by [deleted] »

foldr wrote: post #9

>The other day I saw this Twitter thread. Briefly: GPT knows many human languages, InstructGPT is GPT plus some finetuning in English. Then they fed InstructGPT requests in some other human language, and it carries them out, following the English-language finetuning. >And I thought: so what? Isn’t this expected behaviour? Then a friend pointed out that this is only confusing if you think InstructGPT doesn’t understan…

[deleted]

Re: And yet It Understands

#37

Post by MrScruff »

This article lines up well with my feelings on the matter. In general, people seem to understate the emergent behaviours of ML models, while overstating the uniqueness of human intelligence. I think a lot of this is down to the fact that although both systems exhibit a form of intelligence, they’re very different. LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat.

So it’s not ‘human like intelligence’ but it is a form of intelligence and the reality is no one would have predicted the behaviours we are seeing. So it seems silly to pretend we can know for certain how it achieves its results.

For human intelligence, do we assume cave men had theory of mind at the level of modern day humans? Or did language have to develop first? Our intelligence is built on previous generations, and most of us just ‘interpolate’ within that to a large extent. We behave on occasion like ‘stochastic parrots’ too, mindlessly repeating some new term or phrase we’ve started hearing on Hacker News (why? It just felt like the ‘right thing’ to say).

Human intelligence is the working example that combinations of atoms built into large networks have emergent properties. I’m sure our artificial networks won’t behave qualitatively like the human one as they continue to develop, but I think the burden of proof is on those that suggest we can know what ultimately is and isn’t possible.

Re: And yet It Understands

#38

Post by zetalyrae »

smitty1e wrote: post #24

> But nobody knows how GPT works. They know how it was trained, because the training scheme was designed by humans, but the algorithm that is executed during inference was not intelligently designed but evolved, and it is implicit in the structure of the network, and interpretability has yet to mature to the point where we can draw a symbolic, abstract, human-readable program out of a sea of weights. I object. ChatGP…

I don't see the relevance.

Natural intelligence has not been understood because if our brains were simple enough that we could understand them, we would be so simple we couldn't. This explains why it has not been reproduced.

>There is no repeatable experiment going from inorganic matter to self-aware, self-replicating life.

You can do the RNA world experiments in a lab, it would just take a lot of time and a lot of primordial soup, but eventually you would observe abiogenesis. I also don't see how abiogenesis and biology are relevant.

>capable of doing the impossible, e.g. predicting the weather with fidelity substantially into the future.

Thankfully this is not a measure of intelligence, since we can't do that either.

Re: And yet It Understands

#39

Post by ben_w »

foldr wrote: post #18
zetalyrae wrote: post #14

Earlier quoted context omitted.

This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.

Sure, but the system should then be able to show its working and explain how it derived the incorrect result. If there is other evidence that ChatGPT 'understands' the concept of a prime number, then let's see it. If wrong answers still count because humans sometimes make mistakes, then I guess it won’t be too difficult to construct an impressive mathematical AI. It's very tempting to give these systems the benefit o…

> then I guess it won’t be too difficult to construct an impressive mathematical AI.

Yes, we've had Wolfram Alpha for ages. For me, the biggest problem Wolfram Alpha is that it often doesn't understand the questions, and while I also sometimes get that with ChatGPT, the latter is much much better.

I've not had a chance to play with the plug-in that connects GPT to Wolfram Alpha.

> It's very tempting to give these systems the benefit of the doubt, but that tends to lead to hugely inflated conclusions about their capabilities.

This is an excellent and important point.

I think people who treat it as already being superhuman in the depth (not merely breadth) of each skill are nearly as wrong as those who treat it as merely a souped-up autocomplete.

I've only played with 3 and 3.5 so far, not 4, my impression is that it's somewhere between "a noob" and "genius with Alzheimer's".

A noob at everything at the same time, which is weird because asking a human in English to write JavaScript with all the comments in French and following that up with a request written in German for a description of Cartesian dualism written in Chinese is not something that any human would be expected to do well at, but it can do moderately well at.

Edit: I should probably explicitly add that by my usage of "noob", most people aren't even that good at most things. I might be able to say 你好 if you are willing to overlook a pronunciation so poor it could easily be 尼好 or 拟好 (both of which I only found out about by asking ChatGPT), so I'm sub-noob at Chinese.

Re: And yet It Understands

#40

Post by zetalyrae »

foldr wrote: post #32
zetalyrae wrote: post #21

Earlier quoted context omitted.

On an abstract level, it's obvious that intelligent design, symbolic representations etc. aren't needed to build a mind, because we _evolved_ and evolution is a blind optimizer. But concretely, all the machine learning approaches had many obvious limitations (the volume of data, lack of generalization) until they suddenly didn't, and past a certain scale features of intelligence began to emerge.

>On an abstract level, it's obvious that intelligent design, symbolic representations etc. aren't needed to build a mind, because we _evolved_ and evolution is a blind optimizer. This is playing pretty fast and loose. First of all, I wouldn't lump intelligent design together with the claim that symbolic representations are necessary to account for certain features of human intelligence (such as the classic Fodorian t…

The point is that at no point in evolutionary history did a cognitive scientist sit down and write a bunch of S-expression GOFAI rules for human cognition.
Post reply on HN