Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

141–150 of 231 posts

Re: And yet It Understands

#141

Post by marcosdumay »

kypro wrote: post #76
maister wrote: post #6

I've been thinking a lot about the ability of neural networks to develop understanding and wanted to share my perspective on this. For me it seems absolutely necessary for a NN to develop an understanding of its training data. Take Convolutional Neural Networks (CNNs) used in computer vision, for example. One can observe how the level of abstraction increases in each layer. It starts with detecting brightness transit…

I mean, isn't this the whole point of large + deep NNs? To model complex relationships in data? It's odd so many people seem to deny this with GPT and try to trivialise what it does by saying, "it just predicts the next word". This idea that GPT only works at the level of words and develops no deeper understanding of the concepts in language seems silly given its behaviour. And at the very least it's not what we obse…

> This isn't knowledge regurgitation.

What makes you say that?

Why do you think it's "reasoning" an answer, instead of looking up that people being grounded makes them frustrated?

Re: And yet It Understands

#142

Post by detrites »

Certhas wrote: post #86

Human intelligence evolved with the goal to survive and procreate. GPT intelligence evolved to mimick human speech. Both tasks require a conceptual understanding of the world humans inhabit, but otherwise the two tasks that gave rise to these intelligences are utterly different. We should expect these intelligences to be completely different.

Such things can be expected to be well-encoded in examples of the language. GPT doesn't only mimic human speech, it's built upon an absolutely massive set of the probabilities of speech that is likely to follow other speech. So, our human stories.

Re: And yet It Understands

#143

Post by _dain_ »

marcosdumay wrote: post #133
_xnmw wrote: post #127

I still hold that it doesn't "understand". Even if it answered all questions perfectly, stopped making mistakes, and produced fully working programs better than the best crack developer teams, that still doesn't mean it "understands". "Understanding" is not an output, it's a process, that is sometimes (but not always) measured by its output.

If it stopped making mistakes and produced complete fully working programs, no, there would be no way to say it doesn't understand. Yes, "understanding" is a process, but it's not well defined. And anyway, if it's a requirement for those things, and the AI did those things, then the only possibility is that the AI has this process in some way. But well, our current AIs do not produce complete programs, nor fully work…

What makes you so confident? Did you accurately predict in 2021 what the SOTA LLM capabilities would be in 2023?

Re: And yet It Understands

#144

Post by quonn »

rain1 wrote: post #103
quonn wrote: post #49

I think it would be useful for some HN readers to get some basic philosophy training, specifically on the philosophy of mind. I asked myself many of these questions around 2005 or something and started to read up and there are many experiments that have been done and ChatGPT does not change much for the theory. It is interesting because of it‘s possible economic impact etc. Not because because of any supposed moral c…

Do you have any recommendations on where to learn this? Ideally some online course or just a single really good textbook to study?

There was a German one by Thomas Metzinger: Grundkurs Philosophie des Geistes - Gesamtwerk. It‘s a broad introduction with a historical perspective.

https://www.thalia.de/shop/home/artikeldetails/A1000850443

He also published a popular science book I have not read but that‘s available in English: The Ego Tunnel

https://www.lehmanns.de/shop/geisteswissenschaften/11974733-...

Re: And yet It Understands

#145

Post by civilized »

famouswaffles wrote: post #123
civilized wrote: post #119

Earlier quoted context omitted.

The issue I'm pointing to here doesn't have anything to do with deep understanding of primes built over years of education etc etc. All I'm saying is that ChatGPT doesn't know that 2^2 = 4 means 2^2 * a = 4 * a for any a. ChatGPT is often good at understanding patterns involving the substitution of one string for another. So you might hope that it could do well in a case like this. But it doesn't really. It is aware…

The point I'm making is that chatGPT doesn't know that because the model it has of arithmetic is faulty. It's really as simple as that. The primes thing was just an illustrative example. You want it to do large scale arithmetic with high accuracy? Describe arithmetic as an algorithm to be performed on two numbers. https://arxiv.org/abs/2211.09066 It's not about chatGPT not understanding anything. It's about chatGPT n…

I get what you're saying. I wonder if you are fully following through the implications.

If GPT were generally intelligent, we shouldn't need to devote a special research project to teaching it math. We could just throw a math textbook at it, explanations and worked examples, and it would figure it out from there. Almost certainly its training data contains a great deal of such material already. That this doesn't work suggests its mental architecture is insufficient to grasp what it's been told. (Note that people are not advantaged with any specialized symbolic representation of numbers, like the integer data type a computer has. We manipulate numerical symbols as text, same as the AI.)

It's all well and good that it can improve when a special effort is made, but it sounds like even with that special effort, it still doesn't show the level of competence one would expect from a human-level intelligence with access to virtually infinite, untiring silicon computational resources.

GPT has access to abundant materials to learn the laws of arithmetic from, and it can tell you what they are (because it memorizes everything) but it isn't really understanding what it's learned. That points to a shortcoming of the architecture that won't be solved by merely throwing more data at it.

Re: And yet It Understands

#146

Post by alwaysbeconsing »

ilaksh wrote: post #136
PoignardAzur wrote: post #64

Earlier quoted context omitted.

> Sydney is not supposed to give coherent 3-part messages using them, right? Right, that's the "what the actual fuck" part. This raises some very interesting questions about how Sydney generates its output and the input suggestion. Presumably the LLM is given a prompt like "First generate an answer to the previous text, then generate three input suggestions for the user"; also, the fact that Sydney "hides" the messag…

But it is aligned with deeper human values which is probably what we really want rather than blindly following the instruction to the letter in a life-and-death situation.

This instance (Sydney) is, but what about a differently-configured instance?

Re: And yet It Understands

#147

Post by abecedarius »

espadrine wrote: post #85
the_gipsy wrote: post #69

Earlier quoted context omitted.

Maybe it is human like intelligence already. Maybe our internal monologue is just a better trained and refined ChatGPT. And maybe that is all the magic that is necessary for this holy grail of consciousness, there is no quantum brain, no nothing. Just a stream of the next word that says that we are there, therefore we are. That is what scares me.

> Maybe it is human like intelligence already. It objectively is not. First, intelligence (human or otherwise) is typically defined as the sum of value functions of a given policy across all states, divided by an exponential of the complexity of the environment[0] (page 23). The GPT-4 technical report[1] specifically points out that it is unable to make money when given an initial amount and AWS access (appendix H §…

I wouldn't report that bit from the appendix without noting the test was on an earlier version of the model in development.

Re: And yet It Understands

#148

Post by grey-area »

glenstein wrote: post #65
raincole wrote: post #42

Earlier quoted context omitted.

I don't know. I've been using Copilot, ChatGPT, and Bing Chat intensively in the past month. So far I still think the metaphor "you can’t get to the Moon by piling up chairs" aged very well. > A criticism that would make more sense would be something along the lines of "piling up colors you won't get you any closer to the Moon", since colors aren't even the right kind of thing, and you can't aggregate them in a way t…

>means that even chairs and rockets share some attributes (helping you get higher), the difference between them is still qualitative, not quantitve. I don't think so, because the needed 'quality' is the ability to traverse space. So I don't think I agree that the qualitative piece is missing. Perhaps the moon example is helpful here because the real solution, a rocket ship, uses propulsion rather than sheer mass, and…

Even if you had enough chairs, they are not strong enough.

Re: And yet It Understands

#149

Post by Animats »

There's more of a model inside large language models than was previously thought. How much of a model? Nobody seems to know. There was that one result where someone found what looked like an Othello board in the neuron state.

Someone wrote, below: > We know the basic architecture of large language models, but hardly anything about how they calculate anything specific. That’s the mystery. It will take research, not casual tinkering.

Yes. This is an unexpected situation. Understanding how these things work is way behind making them work. Which is a big problem, since they make up plausible stuff when they don't understand.

Re: And yet It Understands

#150

Post by famouswaffles »

civilized wrote: post #145
famouswaffles wrote: post #123

Earlier quoted context omitted.

The point I'm making is that chatGPT doesn't know that because the model it has of arithmetic is faulty. It's really as simple as that. The primes thing was just an illustrative example. You want it to do large scale arithmetic with high accuracy? Describe arithmetic as an algorithm to be performed on two numbers. https://arxiv.org/abs/2211.09066 It's not about chatGPT not understanding anything. It's about chatGPT n…

I get what you're saying. I wonder if you are fully following through the implications. If GPT were generally intelligent, we shouldn't need to devote a special research project to teaching it math. We could just throw a math textbook at it, explanations and worked examples, and it would figure it out from there. Almost certainly its training data contains a great deal of such material already. That this doesn't work…

Are humans not generally intelligent ?. Since when has been the answer to "not good at math" been chuck a textbook at it ? You would have limited success doing this with people.

Do people not explain things they don't fully understand?

Understanding is not binary.

This is kind of problem I keep seeing. Expectations and post shifting have grown so much that a significant chunk of the human population wouldn't even pass so called General Intelligence requirements.

Post reply on HN