Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

41–50 of 231 posts

Re: And yet It Understands

#41

Post by bmacho »

foldr wrote: post #12
Etheryte wrote: post #11

Earlier quoted context omitted.

This is a common misconception. ChatGPT is not supposed to be good at this — it's a language model, not a maths model or data science model or whatnot. This is exactly why they have plugins, such as the one for Wolfram Alpha.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

Or, it has a deep conceptual understanding of its input data, but, it works similar to humans, and during its reasoning it pulls out some facts from memory or hand-waving, which can be wrong.

No mathematician would make this exact mistake because it looks ridiculous, but for example I made similar mistakes multiple times, claiming low numbers prime or not prime wrongly during a reasoning. Especially when working at night.

Re: And yet It Understands

#42

Post by raincole »

glenstein wrote: post #16

>I was a deep learning skeptic. I doubted that you could get to intelligence by matrix multiplication for the same reason you can’t get to the Moon by piling up chairs I've always been fascinated by this example. I've also heard it referred to as climbing a tree won't get you to the Moon. Because, for some reason, people think that's an argument against the possibility of getting to the Moon when it's actually a prof…

I don't know. I've been using Copilot, ChatGPT, and Bing Chat intensively in the past month. So far I still think the metaphor "you can’t get to the Moon by piling up chairs" aged very well.

> A criticism that would make more sense would be something along the lines of "piling up colors you won't get you any closer to the Moon", since colors aren't even the right kind of thing, and you can't aggregate them in a way that gets you spatially closer. Because that at least does not concede the fundamental relationship of spatial traverseability.

No. I think you misunderstood what the chair metaphor is trying to say. It makes perfect sense, and if you replaced it with colors it would stop making sense.

"You can’t get to the Moon by piling up chairs" means that even chairs and rockets share some attributes (helping you get higher), the difference between them is still qualitative, not quantitve. The problem isn't "we don't have enough chairs."

"You can't get to the Moon by piling up colors" makes zero sense because colors and rockets share so few attributes and the metaphor falls flat.

Is it true for AI? I don't know, and at this point, anyone who claims they knows for sure, is lying. But I don't think my experience with ChatGTP (GPT4 included) and Bing Chat disputes it.

Re: And yet It Understands

#43

Post by zetalyrae »

civilized wrote: post #30
zetalyrae wrote: post #14

Earlier quoted context omitted.

This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.

But it's much worse than that for ChatGPT. A person would know that their mental math division of large numbers is unreliable, but if you ask ChatGPT about the reliability of its answers, it is uniformly confident about almost everything. It has no grasp of the difference between the things it "knows" which are true and the things it "knows" which may be false. And this is the error I notice people very consistently…

The failures don't refute the successes. Anything and anyone can fail, you don't get intelligent output by chance. If it makes mistakes we wouldn't make, on obvious things, it is because it is an alien form of intellingece.

RLHF and the tokenizer together explain many of the more common failure modes.

Re: And yet It Understands

#44

Post by foldr »

zetalyrae wrote: post #40
foldr wrote: post #32

Earlier quoted context omitted.

>On an abstract level, it's obvious that intelligent design, symbolic representations etc. aren't needed to build a mind, because we _evolved_ and evolution is a blind optimizer. This is playing pretty fast and loose. First of all, I wouldn't lump intelligent design together with the claim that symbolic representations are necessary to account for certain features of human intelligence (such as the classic Fodorian t…

The point is that at no point in evolutionary history did a cognitive scientist sit down and write a bunch of S-expression GOFAI rules for human cognition.

Of course not, but who is claiming that this happened? I'm not sure I can think of even single person who is both (i) an intelligent design advocate and (ii) explicitly committed to a symbolic model of human cognitive capacities. This seems like a straw man. You might as well say that at no point in evolutionary history did an embryologist ever write down a plan for the eight developmental stages of a human fetus.

Re: And yet It Understands

#45

Post by tjr »

I keep being reminded of Paul Graham's "plan for spam", in that he devised a simple statistical evaluator, and was surprised that it worked so well to distinguish ham from spam.

These AI tools have been trained on a great deal of written language artifacts and exhibit a surprising level of what appears to be concept understanding. Perhaps the real surprise is that language conveys concepts better than we previously thought?

Re: And yet It Understands

#47

Post by maxdoop »

The gap between AI “acceptance / exploration” and “AI dismissal” continues to widen.

Right now, the top post on HN is about how ChatGPT is “a glorified text prediction program.”

Right under that post is this post.

Re: And yet It Understands

#48

Post by ElFitz »

foldr wrote: post #32
zetalyrae wrote: post #21

Earlier quoted context omitted.

On an abstract level, it's obvious that intelligent design, symbolic representations etc. aren't needed to build a mind, because we _evolved_ and evolution is a blind optimizer. But concretely, all the machine learning approaches had many obvious limitations (the volume of data, lack of generalization) until they suddenly didn't, and past a certain scale features of intelligence began to emerge.

>On an abstract level, it's obvious that intelligent design, symbolic representations etc. aren't needed to build a mind, because we _evolved_ and evolution is a blind optimizer. This is playing pretty fast and loose. First of all, I wouldn't lump intelligent design together with the claim that symbolic representations are necessary to account for certain features of human intelligence (such as the classic Fodorian t…

I might be wrong but I think it’s more of a "It’s obvious you don’t need to have fingers to build a hand that has fingers" than building a fingerless functional equivalent to a hand.

Re: And yet It Understands

#49

Post by quonn »

I think it would be useful for some HN readers to get some basic philosophy training, specifically on the philosophy of mind. I asked myself many of these questions around 2005 or something and started to read up and there are many experiments that have been done and ChatGPT does not change much for the theory. It is interesting because of it‘s possible economic impact etc. Not because because of any supposed moral concern for the software itself of which there are none at this point.

I see people here constantly mixing intelligence and conciousness and that‘s really the most basic destinction everyone should be able to make.

Re: And yet It Understands

#50

Post by rain1 »

maxdoop wrote: post #47

The gap between AI “acceptance / exploration” and “AI dismissal” continues to widen. Right now, the top post on HN is about how ChatGPT is “a glorified text prediction program.” Right under that post is this post.

we can't both be wrong!
Post reply on HN