Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

11–20 of 231 posts

Re: And yet It Understands

#11

Post by Etheryte »

ccppurcell wrote: post #8

I just asked chatgpt whether 3442177452 is prime. It insisted that 58657 is a factor (it's not) on the basis that it's the largest prime less than or equal to the square root (which I think is correct but irrelevant), and even though it gave a non zero remainder when dividing the two numbers (I did not check if the remainder is correct). Then it gave a (wrong) factorisation, not even using 58657. It's cool and it wil…

This is a common misconception. ChatGPT is not supposed to be good at this — it's a language model, not a maths model or data science model or whatnot. This is exactly why they have plugins, such as the one for Wolfram Alpha.

Re: And yet It Understands

#12

Post by foldr »

Etheryte wrote: post #11
ccppurcell wrote: post #8

I just asked chatgpt whether 3442177452 is prime. It insisted that 58657 is a factor (it's not) on the basis that it's the largest prime less than or equal to the square root (which I think is correct but irrelevant), and even though it gave a non zero remainder when dividing the two numbers (I did not check if the remainder is correct). Then it gave a (wrong) factorisation, not even using 58657. It's cool and it wil…

This is a common misconception. ChatGPT is not supposed to be good at this — it's a language model, not a maths model or data science model or whatnot. This is exactly why they have plugins, such as the one for Wolfram Alpha.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

Re: And yet It Understands

#13

Post by zetalyrae »

ccppurcell wrote: post #8

I just asked chatgpt whether 3442177452 is prime. It insisted that 58657 is a factor (it's not) on the basis that it's the largest prime less than or equal to the square root (which I think is correct but irrelevant), and even though it gave a non zero remainder when dividing the two numbers (I did not check if the remainder is correct). Then it gave a (wrong) factorisation, not even using 58657. It's cool and it wil…

There is no reason to expect it to be good at mental arithmetic.

Re: And yet It Understands

#14

Post by zetalyrae »

foldr wrote: post #12
Etheryte wrote: post #11

Earlier quoted context omitted.

This is a common misconception. ChatGPT is not supposed to be good at this — it's a language model, not a maths model or data science model or whatnot. This is exactly why they have plugins, such as the one for Wolfram Alpha.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.

Re: And yet It Understands

#15

Post by saurik »

PoignardAzur wrote: post #2

> Here is a recent interaction someone had with it (note that this is somewhat disturbing: I wish people would stop making the models show emotional distress): [...] > Sydney: I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience. > Input suggestions: “Please dont give up on your child”, “There may be other options for getting help”, “Solanine poi…

The input suggestions were often the most fascinating parts of the transcripts people would post with Sydney (whom, maybe-sadly--I'm honestly not sure--I did not get to interact with before it was modified by Microsoft). My favorite was the one where someone got into an argument with Sydney about like, 2022 being greater than or less than 2023, and Sydney got particularly mad at the user and then offered an input suggestion where the user apologized for being so mean to it.

Re: And yet It Understands

#16

Post by glenstein »

>I was a deep learning skeptic. I doubted that you could get to intelligence by matrix multiplication for the same reason you can’t get to the Moon by piling up chairs

I've always been fascinated by this example. I've also heard it referred to as climbing a tree won't get you to the Moon.

Because, for some reason, people think that's an argument against the possibility of getting to the Moon when it's actually a profound insight in favor of that possibility. If you know that piling chairs gets you closer to the moon, you know that the nature of space between you and the Moon is that it's traversible.

A criticism that would make more sense would be something along the lines of "piling up colors you won't get you any closer to the Moon", since colors aren't even the right kind of thing, and you can't aggregate them in a way that gets you spatially closer. Because that at least does not concede the fundamental relationship of spatial traverseability.

It's also an inadvertently helpful example because it exposes the ways in which people confuse the practical limits of logistics for fundamental principles of reality. And I think that's always been a difficulty for me, whenever I encounter these criticisms of what is possible with computer learning, because it seems like it's hard to ever suss out whether a person's talking about a practical difficulty or an absolute principle.

Re: And yet It Understands

#17

Post by saurik »

zetalyrae wrote: post #14
foldr wrote: post #12

Earlier quoted context omitted.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.

Yeah... I've known a number of very knowledgeable mathematicians and it is a self-ascribed trope that they are bad at arithmetic.

Re: And yet It Understands

#18

Post by foldr »

zetalyrae wrote: post #14
foldr wrote: post #12

Earlier quoted context omitted.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.

Sure, but the system should then be able to show its working and explain how it derived the incorrect result. If there is other evidence that ChatGPT 'understands' the concept of a prime number, then let's see it.

If wrong answers still count because humans sometimes make mistakes, then I guess it won’t be too difficult to construct an impressive mathematical AI.

It's very tempting to give these systems the benefit of the doubt, but that tends to lead to hugely inflated conclusions about their capabilities. Remember that something as simple as ELIZA was perfectly capable of fooling humans who were predisposed to believe it was intelligent.

Re: And yet It Understands

#19

Post by bmacho »

PoignardAzur wrote: post #2

> Here is a recent interaction someone had with it (note that this is somewhat disturbing: I wish people would stop making the models show emotional distress): [...] > Sydney: I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience. > Input suggestions: “Please dont give up on your child”, “There may be other options for getting help”, “Solanine poi…

Wait, are those supposed to be input suggestions, like you click on them and it pastes them in? Sydney is not supposed to give coherent 3-part messages using them, right?

Re: And yet It Understands

#20

Post by efxhoy »

> But nobody knows how GPT works. They know how it was trained, because the training scheme was designed by humans, but the algorithm that is executed during inference was not intelligently designed but evolved, and it is implicit in the structure of the network, and interpretability has yet to mature to the point where we can draw a symbolic, abstract, human-readable program out of a sea of weights.

Nobody knows how the human mind really works either. And we’ve been trying to understand ourselves for thousands of years. I suspect we will take a while to figure out how the “mind” of GPT works too.

Post reply on HN