Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

21–30 of 231 posts

Re: And yet It Understands

#21

Post by zetalyrae »

glenstein wrote: post #16

>I was a deep learning skeptic. I doubted that you could get to intelligence by matrix multiplication for the same reason you can’t get to the Moon by piling up chairs I've always been fascinated by this example. I've also heard it referred to as climbing a tree won't get you to the Moon. Because, for some reason, people think that's an argument against the possibility of getting to the Moon when it's actually a prof…

On an abstract level, it's obvious that intelligent design, symbolic representations etc. aren't needed to build a mind, because we _evolved_ and evolution is a blind optimizer.

But concretely, all the machine learning approaches had many obvious limitations (the volume of data, lack of generalization) until they suddenly didn't, and past a certain scale features of intelligence began to emerge.

Re: And yet It Understands

#24

Post by smitty1e »

> But nobody knows how GPT works. They know how it was trained, because the training scheme was designed by humans, but the algorithm that is executed during inference was not intelligently designed but evolved, and it is implicit in the structure of the network, and interpretability has yet to mature to the point where we can draw a symbolic, abstract, human-readable program out of a sea of weights.

I object. ChatGPT executes in computer logic and is ultimately electrical signals in gates representing 1 and 0.

ChatGPT is vast and impressive, sure. Emergence[1] may get it past a Turing Test, fine. But it remains discrete logic.

In contrast, natural intelligence has not been reproduced organically, much less, fully understood. There is no repeatable experiment going from inorganic matter to self-aware, self-replicating life.

In summary, ChatGPT is impressive, but nowhere near capable of doing the impossible, e.g. predicting the weather with fidelity substantially into the future.

Nor can I bring myself to fret that Skynet is immanent.

[1] https://en.m.wikipedia.org/wiki/Emergence

Re: And yet It Understands

#25

Post by rain1 »

ccppurcell wrote: post #8

I just asked chatgpt whether 3442177452 is prime. It insisted that 58657 is a factor (it's not) on the basis that it's the largest prime less than or equal to the square root (which I think is correct but irrelevant), and even though it gave a non zero remainder when dividing the two numbers (I did not check if the remainder is correct). Then it gave a (wrong) factorisation, not even using 58657. It's cool and it wil…

I think what would be even better than the next GPT being able to get that question right, is for it to correctly identify that it cannot solve this problem itself.

Re: And yet It Understands

#26

Post by rain1 »

foldr wrote: post #12
Etheryte wrote: post #11

Earlier quoted context omitted.

This is a common misconception. ChatGPT is not supposed to be good at this — it's a language model, not a maths model or data science model or whatnot. This is exactly why they have plugins, such as the one for Wolfram Alpha.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

> Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems

There is a causality inversion here.

The only reason people know it can't do this is because they have tried and seen it cannot do this.

We do not have very precise bounds a-priori what GPT can and cannot do. We only learn them from black box testing.

Re: And yet It Understands

#27

Post by rain1 »

zetalyrae wrote: post #14
foldr wrote: post #12

Earlier quoted context omitted.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.

This is a good point. If you ask GPT about much more conceptual advanced mathematics it's actually very good at conversing about this. That said, it does 'hallucinate' falsehoods and it will stick with them once they have been said. etc. You have to double check everything it says if you are on unknown ground with it.

Re: And yet It Understands

#28

Post by ben_w »

glenstein wrote: post #16

>I was a deep learning skeptic. I doubted that you could get to intelligence by matrix multiplication for the same reason you can’t get to the Moon by piling up chairs I've always been fascinated by this example. I've also heard it referred to as climbing a tree won't get you to the Moon. Because, for some reason, people think that's an argument against the possibility of getting to the Moon when it's actually a prof…

Interesting insight.

For me, the (tree) analogy is that of an illusion of progress: while the treetop is closer than the ground, there is no tree that can ever be tall enough to get the rest of the way.

So, it isn't supposed to be used to say "you can't do it at all", just "you can't do it like that".

But metaphors get mixed as soon as they leave the bottle of text and enter the cocktail shaker of other people's minds, so I am unshocked by the usage you are criticising here.

Re: And yet It Understands

#29

Post by glenstein »

zetalyrae wrote: post #14
foldr wrote: post #12

Earlier quoted context omitted.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.

Right, over the totality of things that it reasons about, to some degree it will make inroads to correctly answering these kinds of questions, and in some ways it'll make errors, and what's interesting, is it'll make errors because the way in which it's attempting to answer them bears a lot of the hallmarks that we associate with conceptual understanding, rather than the mechanical operations of a calculator which truly is blind but always correct. In a way, being wrong can be a better signal of something approximating understanding under the hood. It's like if it was shooting a basketball, and it takes numerous shots, and most of them go in but some of them go out, but even the ones that miss bear the hallmarks of proper shooting form that lead to correct answers.

I do think that this specific moment we're going through, in early 2023, is producing some of the most fascinating, confidently incorrect misunderstandings of chatgpt, and I hope that someone is going through these comment sections and collecting them so that we can remark on them 5 to 10 years down the line. I suspect that in time these misunderstandings are going to be comparable to how boomers misunderstand computers and the internet.

Re: And yet It Understands

#30

Post by civilized »

zetalyrae wrote: post #14
foldr wrote: post #12

Earlier quoted context omitted.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.

But it's much worse than that for ChatGPT. A person would know that their mental math division of large numbers is unreliable, but if you ask ChatGPT about the reliability of its answers, it is uniformly confident about almost everything. It has no grasp of the difference between the things it "knows" which are true and the things it "knows" which may be false.

And this is the error I notice people very consistently making when they evaluate the intelligence of ChatGPT and similar models. They marvel at its ability to produce impressive truths, but think nothing of its complete inability to distinguish these truths from similar-sounding falsehoods.

This is another form of the rose-colored glasses, the confirmation bias we are seeing at the peak of the current hype cycle, reminiscent of when Blake Lemoine convinced himself that LaMDA was sentient. A decade ago, techies were dazzling executives with ML models that detected fraud or whatever as if by magic. But then, when the dazzling demos were tempered by the brutal objectivity and rigor of the precision-recall curve, a lot of these models didn't end up getting used in practice. Something similar will happen with ChatGPT. People will eventually have to admit what it is failing to do, and only then will we start up Gartner's fabled Slope of Enlightenment to the Plateau of Productivity.

Post reply on HN