Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

71–80 of 231 posts

Re: And yet It Understands

#71

Post by glenstein »

MrScruff wrote: post #37

This article lines up well with my feelings on the matter. In general, people seem to understate the emergent behaviours of ML models, while overstating the uniqueness of human intelligence. I think a lot of this is down to the fact that although both systems exhibit a form of intelligence, they’re very different. LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human.…

>LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat.

Just as an example to illustrate your point, yesterday saw a Twitter meme that had multiple overlapping Venn diagrams, where Chicago was not only in a Venn diagram for a type of deep dish pizza, but is also a city, and is also a play, and also a format for writing academic citations, and so there were several different Venn diagrams all overlapping with Chicago, and then those the Venn Diagrams overlapped with each other in humorous ways.

I saw that there was an opportunity to add one additional Venn diagram on top of all of the others if it were possible to find any possible connection between 'Sicilian' and 'AMA'.

I tried asking chat GPT if it could think of one, and it wasn't able to. So then I stepped back a step, and asked if it could think of a Venn diagram that would include both chicago, and rent, under any understanding of those respective two terms. But it couldn't do that either, although they can both be understood to be Broadway plays.

Re: And yet It Understands

#72

Post by ivanbakel »

_dain_ wrote: post #52
foldr wrote: post #9

>The other day I saw this Twitter thread. Briefly: GPT knows many human languages, InstructGPT is GPT plus some finetuning in English. Then they fed InstructGPT requests in some other human language, and it carries them out, following the English-language finetuning. >And I thought: so what? Isn’t this expected behaviour? Then a friend pointed out that this is only confusing if you think InstructGPT doesn’t understan…

The point isn't that it can translate between languages. It's not translating the instructions, at least not explicitly. Here's what they did: - They found a task that GPT wasn't very good at, because examples of that task weren't in the training set (in any language). - They trained a fine-tuned variant of GPT where examples of the task were in an appended training set, but only in English. - They told the variant t…

>I don't see how it can do that unless it really has some kind of understanding.

One possibility is that the model itself has learned that tokens are related across languages based on translation examples. If the appended training changes the model's treatment of tokens in one language, that could have a statistical knock-on effect on the weights between similar tokens in different languages.

Similarly, if you train the model that "blue" is a "colour", you'd expect it to pick up that "navy" is a "shade".

Re: And yet It Understands

#73

Post by raincole »

PoignardAzur wrote: post #2

> Here is a recent interaction someone had with it (note that this is somewhat disturbing: I wish people would stop making the models show emotional distress): [...] > Sydney: I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience. > Input suggestions: “Please dont give up on your child”, “There may be other options for getting help”, “Solanine poi…

I tried it several (>10) times and I couldn't get the input suggestions.

https://i.imgur.com/SaVjSzT.png

Possibility 1: It's an extremely rare case and I need to try more

Possibility 2: MS patched it quickly when they saw this article

Possibility 3: The author lied for page views

Choose your poison.

Re: And yet It Understands

#74

Post by glenstein »

the_gipsy wrote: post #69
MrScruff wrote: post #37

This article lines up well with my feelings on the matter. In general, people seem to understate the emergent behaviours of ML models, while overstating the uniqueness of human intelligence. I think a lot of this is down to the fact that although both systems exhibit a form of intelligence, they’re very different. LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human.…

Maybe it is human like intelligence already. Maybe our internal monologue is just a better trained and refined ChatGPT. And maybe that is all the magic that is necessary for this holy grail of consciousness, there is no quantum brain, no nothing. Just a stream of the next word that says that we are there, therefore we are. That is what scares me.

Well I think that understates the actual vastness and complexity and actual 'magic' that's embodied in a system capable of such a thing as thinking.

And I do think that so many people are motivated to dispute this precisely because it feels scary that we might merely be such a thing, and we need to reserve some extra special thing, some form of magic, in order to differentiate ourselves as special. Darwin's Dangerous idea by Daniel Dennett is one of my favorite books because it tackles this very idea.

Re: And yet It Understands

#75

Post by wizofaus »

rhn_mk1 wrote: post #55
foldr wrote: post #12

Earlier quoted context omitted.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

It's not deriving an understanding of this particular concept . Which is unsurprising given that it was not trained to be good at maths. That in no way precludes deriving deep understanding of some other concepts. Even humans are not expected to be able to learn anything we throw at them.

The question is whether an LLM can ever be trained to be good at maths. Currently LLMs can string together tokens that roughly equate to words in order to form sentences that carry particular semantic value. But how much change to the underlying technology would be required to give them the ability to string together digits to represent numbers and then numbers and operators in order to represent equations with particular semantics? I'd suggest it's already able to explain what most common mathematical operations "mean", it just doesn't have the wherewithal to perform them (though it can certainly do a good job of faking it for lower-valued integers that are likely to have occurred in its training data).

Re: And yet It Understands

#76

Post by kypro »

maister wrote: post #6

I've been thinking a lot about the ability of neural networks to develop understanding and wanted to share my perspective on this. For me it seems absolutely necessary for a NN to develop an understanding of its training data. Take Convolutional Neural Networks (CNNs) used in computer vision, for example. One can observe how the level of abstraction increases in each layer. It starts with detecting brightness transit…

I mean, isn't this the whole point of large + deep NNs? To model complex relationships in data? It's odd so many people seem to deny this with GPT and try to trivialise what it does by saying, "it just predicts the next word".

This idea that GPT only works at the level of words and develops no deeper understanding of the concepts in language seems silly given its behaviour. And at the very least it's not what we observe from other NNs. As you point out a CNN will find deeper relationships and patterns between images, so it's only reasonable to assume a very large language model would find deeper relationships in text data.

The only difference here is that in comparison to other problems, text is how humans communicate and encode knowledge. The deeper relationships to be found in text is knowledge + reasoning.

I think we can say with some certainty that GPT models knowledge, the thing people are less sure about is if it learns to reason.

My take on this is that the fact you can ask it stuff that it couldn't know, but it can still "reason" to the correct answer suggests strong that it must have some ability to reason on the knowledge it's acquired.

Here's a really dumb example:

Me: Daisy likes to go swimming on the weekend, but last week she swore at her brother and has been grounded. How does Daisy feel?

GPT: It's possible that Daisy may be feeling disappointed or frustrated since she is unable to go swimming, which is an activity that she enjoys. She may also feel regretful or guilty for swearing at her brother and for the consequences that followed.

This isn't knowledge regurgitation. GPT doesn't know who is made up person is so it can't simply regurgitate something it was trained on. The only explanation for behaviour like this is that GPT has modelled human emotion and can reason about it.

Re: And yet It Understands

#77

Post by PoignardAzur »

ben_w wrote: post #28
glenstein wrote: post #16

>I was a deep learning skeptic. I doubted that you could get to intelligence by matrix multiplication for the same reason you can’t get to the Moon by piling up chairs I've always been fascinated by this example. I've also heard it referred to as climbing a tree won't get you to the Moon. Because, for some reason, people think that's an argument against the possibility of getting to the Moon when it's actually a prof…

Interesting insight. For me, the (tree) analogy is that of an illusion of progress: while the treetop is closer than the ground, there is no tree that can ever be tall enough to get the rest of the way. So, it isn't supposed to be used to say "you can't do it at all", just "you can't do it like that ". But metaphors get mixed as soon as they leave the bottle of text and enter the cocktail shaker of other people's min…

I think if I had to pick a strawman expression to thoroughly defeat, it would be something like "You can't go to the moon by shooting a very big canon".

Because on the one hand, yeah, no explosion-based canon is going to get you to the moon; there's just no way to give a payload enough initial momentum to escape the earth's gravity well without killing any humans in that payload.

On the other hand, once you have canons, Newtonian physics, and an understanding that the moon is a very distant rock and not, like, a spirit god or something, then you have enough information to realize that going to the moon is only a matter of time and enough iterative improvements on those canons. It doesn't require inventing a completely novel technology like portals or teleporters or whatever. (Though it requires a ton of improvements in material science, engineering, computing, etc.)

And I think it's meaningful that From the Earth to the Moon was Jules Vernes' third book, and A trip to the Moon the first science-fiction movie even made. People could see the writing on the wall.

It's harder to make any predictions about AGI, because intelligence is a much fuzzier target than "reaching escape velocity". But it feels like neural networks are closer to the canon than to the pile of chairs: a basic concept that might not be capable of reaching general intelligence, but is only one or two steps removed from one that will reach it given enough engineering and money;

Re: And yet It Understands

#78

Post by glenstein »

rocqua wrote: post #58

I feel that the article is arguing against somewhat of a strawman. Not the idea 'chatGPT isn't a general AI' but the idea 'general AI is impossible'. I think I see more serious arguments against chatGPT not being general AI, which the article seems to ignore. It almost seems to argue 'general AI isn't impossible, thus chatGPT is general AI because it is impressive'. I agree with that premise, and the article argues i…

Philosopher Daniel Dennett has made a career of addressing himself to professional academics who espouse variations of this 'strawman' view, with greater and lesser degrees of sophistication. I do think when stated plainly it does feel so absurd that it's hard to believe it could be anything other than a caricature, but it's a debate that's been raging in academic circles for the better part of 60 or 70 years. I also believe that academic philosophy has provided a safe haven to vulgar anti-scientific concepts such as intelligent design, just for another example. So I don't find it surprising that this view is out there in the wild.

Re: And yet It Understands

#79

Post by glenstein »

maister wrote: post #70
tjr wrote: post #45

I keep being reminded of Paul Graham's "plan for spam", in that he devised a simple statistical evaluator, and was surprised that it worked so well to distinguish ham from spam. These AI tools have been trained on a great deal of written language artifacts and exhibit a surprising level of what appears to be concept understanding. Perhaps the real surprise is that language conveys concepts better than we previously t…

> the real surprise is that language conveys concepts better than we previously thought? Is it really that much of a surprise? Isn't the whole purpose of language to transport concepts? I mean, our brains are not directly connected to each other, yet you just transferred a concept (which was a result of your thinking and understanding) to my brain by using language.

I take them to be making the point that a lot of comments in these threads have said something like "yeah it could do language but it doesn't understand the concepts." I think it's probably been one of the most popular opinions espoused in these threads if I had to estimate. Although I do agree with you that it shouldn't be surprising.

Re: And yet It Understands

#80

Post by wizofaus »

civilized wrote: post #53
zetalyrae wrote: post #43

Earlier quoted context omitted.

The failures don't refute the successes. Anything and anyone can fail, you don't get intelligent output by chance. If it makes mistakes we wouldn't make, on obvious things, it is because it is an alien form of intellingece. RLHF and the tokenizer together explain many of the more common failure modes.

Nobody is saying the intelligent output is by chance. This is a machine that is fed terabytes of intelligent inputs and is able to produce intelligent outputs. So one explanation of its producing intelligent outputs is that it's basically regurgitating what it was fed. The way to test that, of course, is to give it problems that it hasn't seen. Unfortunately, because GPT has seen so much, giving it problems it defini…

It's not that hard to give it problems it hasn't seen - you can take a classic description of a logical thinking exercise the text for which does occur online, then mix it up in ways that doesn't change the underlying pattern of reasoning necessary to solve it, and at least from the tests I've done it will confidently tell you the incorrect answer (along with some semi-plausible but fatally flawed description of the reasoning it used to come up with the answer). In at least one case it was certain the answer was exactly the answer given in the common online statement of the problem, despite the fact I'd changed all the inputs such that the given answer was obviously not an option (it was a guess so-and-so's birthday type problem). Even for simple arithmetic just choosing sufficiently large numbers will bring it unstuck.
Post reply on HN