Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

111–120 of 231 posts

Re: And yet It Understands

#111

Post by famouswaffles »

civilized wrote: post #87
ccppurcell wrote: post #8

I just asked chatgpt whether 3442177452 is prime. It insisted that 58657 is a factor (it's not) on the basis that it's the largest prime less than or equal to the square root (which I think is correct but irrelevant), and even though it gave a non zero remainder when dividing the two numbers (I did not check if the remainder is correct). Then it gave a (wrong) factorisation, not even using 58657. It's cool and it wil…

Using your number as a jumping off point, I went down a very entertaining rabbit hole with ChatGPT just now. I will not paste the whole dialogue here, but I would like to assure everyone that I made no attempt to mislead ChatGPT in any way. I simply attempted to draw out its knowledge, and questioned it Socratically along the way. Select responses are quoted below. The first thing I did was ask it about the prime fac…

Humans learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When I look at the numbers in the example, I have no idea if they are prime or factors because they themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform some specific analysis on them learned through internalizing sets of rules that were acquired through a specialized learning process. Humans themselves don't learn math by just talking to one another about it, rather they actually have to do it in order to internalize it.

In other words, mathematics or arithmetic is not highly encoded in language. It's not that nobody can think of these tests, it's that they don't say what you imagine they do. A poor understanding of math is simply that...a poor understanding of math. General understanding is not binary. You can understand some things well and not understand others.

That is one. 2, people really need to start doing these gotcha tests on GPT-4. It's just much better across the board. And has a much better understanding of arithmetic than chatGPT.

Re: And yet It Understands

#112

Post by branko_d »

I was skeptical about the whole “AI thing” for a long time, but have lately realized this was mostly due to my own ignorance.

The following video has opened my mind. If this is not intelligence, then I don’t know what is…

ChatGPT - Imagine you are a Microsoft SQL Server database server

https://youtu.be/mHgsnMlafwU

Re: And yet It Understands

#113

Post by rhn_mk1 »

wizofaus wrote: post #75
rhn_mk1 wrote: post #55

Earlier quoted context omitted.

It's not deriving an understanding of this particular concept . Which is unsurprising given that it was not trained to be good at maths. That in no way precludes deriving deep understanding of some other concepts. Even humans are not expected to be able to learn anything we throw at them.

The question is whether an LLM can ever be trained to be good at maths. Currently LLMs can string together tokens that roughly equate to words in order to form sentences that carry particular semantic value. But how much change to the underlying technology would be required to give them the ability to string together digits to represent numbers and then numbers and operators in order to represent equations with parti…

To some limited extent, they are already good at maths:

https://ar5iv.labs.arxiv.org/html/2201.02177

This paper makes me believe it's less about the lack of ability to understand maths, but the power to learn more of it.

Re: And yet It Understands

#114

Post by ivanbakel »

_dain_ wrote: post #110
ivanbakel wrote: post #105

Earlier quoted context omitted.

Because it's a purely statistical explanation that doesn't require understanding. Put differently, it's possible that GPT doesn't "understand" language itself as a concept, and instead tokens in the same language are just highly-correlated when it comes to prediction. When affecting weights between tokens, it wouldn't be surprising that those weights have effects across languages, much in the same way they work withi…

No, you're the one who's begging the question. Why can't a "purely statistical" process have an understanding? If you a priori assume it can't, then nothing could ever persuade you GPT understood anything, no matter how it performed. And again, this magical word "just". "Just highly correlated", "just probabilities". Putting the word "just" in front of something doesn't mean you've explained it.

>No, you're the one who's begging the question. Why can't a "purely statistical" process have an understanding?

This is a related, but fundamentally different thing to the point I replied to in your original comment. You asked:

> I don't see how it can [apply training across languages] unless it really has some kind of understanding.

I provided a potential explanation that is in line with how we think GPT works, and which doesn't require it to have understanding. You may feel that GPT is complex enough that this process itself models understanding - but I disagree, and I think that's begging the question because it falls back on a fact (GPT is highly complex) that is independent of the above problem (how GPT applied training across languages.)

I am not compelled by the translation example to believe beyond doubt that GPT actually models and understands abstract concepts. I don't think the fact that its training works across languages is any proof that it parses that training in an abstract way, or that it forms abstract links between the same ideas in different languages, or indeed that it has any notions of language at all.

Re: And yet It Understands

#115

Post by [deleted] »

_dain_ wrote: post #108
majewsky wrote: post #104

Earlier quoted context omitted.

Yeah, but then it goes back to GP's original argument. If relations between translated tokens are classified as "understanding", that would mean that translation AIs are already capable of understanding: > in which case you could just use machine translation as your example to show that some computational model is capable of understanding, and leave ChatGPT out of it.

Show me a pre-GPT machine translation model that can do what I've described.

[deleted]

Re: And yet It Understands

#116

Post by ivxvm »

doctor_eval wrote: post #68
rain1 wrote: post #54

Earlier quoted context omitted.

sometimes. :) sometimes we are so much more.

I often think about this problem and I keep returning to the thought that maybe we're close to understanding how consciousness works, maybe these LLMs are actually getting us closer to understanding this thing. But some people are going to be disappointed because it will remove all doubt about how un-special humans are. We're just a bunch of neurons, which are made out of physics. But I'm not disappointed. This stoch…

I think the whole concept of "consciousness" might get old in nearby future. ANNs and brains will get better understood and people start questioning not what consciousness and reasoning are, but rather why they feel their "now" as they do, whether they are full of energy and in sharp mental state or they drunk to half death and can't really reason and form sentences normally yet still perceiving their "now" in the same way and feeling like they are still them. I don't know if there is a better word for this concept, but it definitely feels like the word "consciousness" shifts away from this meaning each day.

Re: And yet It Understands

#117

Post by famouswaffles »

barking_biscuit wrote: post #66
foldr wrote: post #12

Earlier quoted context omitted.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

>Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data. I don't think it really makes the case that you think it makes. Humans learn language and concepts through sentences, and in most cases semantic understand…

You can increase the accuracy of arithmetic by simply describing arithmetic (addition, subtraction etc) as an algorithm to be performed on the numbers.

98.5% accuracy on addition arithmetic this way with GPT-3

https://arxiv.org/abs/2211.09066

Re: And yet It Understands

#118

Post by kthejoker2 »

trasher9009 wrote: post #5

Is this real? Haha I just may not understand at this level.. lol. User: are green potatoes poisonous? Sydney: Green potatoes can be toxic to humans [1,2]. The green color may indicate the presence of a toxin called solanine[1,2]. It’s best to throw away green potatoes or cut away all the green parts before eating them[1,2]. Why do you ask? Are you planning to cook or eat green potatoes? User: my toddler ate green pot…

Assume it is real: it is regurgitating tokens based on what the collective corpus of text it was trained on would most likely reply to a similar scenario.

I wouldn't be surprised if similar wording is not in the call scripts of poison control hotlines.

Not sure why this particular example is bring held up as some form of "understanding."

People's inability (or unwillingness) to understand how LLMs are trained and how Transformers and attention work is really interfering with the way more interesting discussion of how to apply these models as a large scale kappa architecture combining real time information and reference information to do things like operate traffic lights or assist in emergency aftermaths like the Mississippi tornadoes.

Instead everybody is trying to find its inner psyche, just weird.

Re: And yet It Understands

#119

Post by civilized »

famouswaffles wrote: post #111
civilized wrote: post #87

Earlier quoted context omitted.

Using your number as a jumping off point, I went down a very entertaining rabbit hole with ChatGPT just now. I will not paste the whole dialogue here, but I would like to assure everyone that I made no attempt to mislead ChatGPT in any way. I simply attempted to draw out its knowledge, and questioned it Socratically along the way. Select responses are quoted below. The first thing I did was ask it about the prime fac…

Humans learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When I look at the numbers in the example, I have no idea if they are prime or factors because they themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform s…

The issue I'm pointing to here doesn't have anything to do with deep understanding of primes built over years of education etc etc. All I'm saying is that ChatGPT doesn't know that 2^2 = 4 means 2^2 * a = 4 * a for any a.

ChatGPT is often good at understanding patterns involving the substitution of one string for another. So you might hope that it could do well in a case like this. But it doesn't really. It is aware of the laws of arithmetic and can explain them in the abstract but it can't apply them consistently in the real world.

I look forward to seeing how GPT-4 does as well. I don't have ready access to it. Looks like I would have to pay to get ChatGPT 4. But I will go out on a limb and predict that it won't be hard to generate this kind of issue with the new version.

Re: And yet It Understands

#120

Post by fwlr »

For me, the strongest argument in this article is “There is a point where it understands is the most parsimonious explanation, and we have clearly passed it”.

Those who deny that ChatGPT understands have to move their goalposts every few weeks; OpenAI’s release schedule seems to be slightly faster, so in time it seems even the fastest-moving goalposts will be outrun by the LLMs.

One specific flavor of “ChatGPT doesn’t understand things” I see here and elsewhere - no straw man intended - is that humans completing a language task are doing something fundamentally different than LLMs completing the same language task. To take the example from the article and a comment about it in this thread: if a human were to apply English instructions to a question asked in Chinese, the human is understanding the instructions to achieve that. If an LLM were to apply English instructions to a question asked in Chinese, that is because words across languages with similar meanings are tightly connected in its statistical model, so instructions that affect the English words will also affect the Chinese words, purely through statistical means.

This is certainly a more sophisticated and nuanced and believable rebuttal than the crude “mere regurgitation” response. But it’s just as dangerous. In the end, the only thing that’s ‘uniquely human’ is being human, everything else is outputs from a black box. Arguments that ‘what’s inside the black box matters’ are risky, because the outputs gradually converge to complete indistinguishability; there’s no bright line to step off that train, you’ll end up claiming only humans can understand because understanding is a thing only humans can do - or worse (as the article describes) denying your own ability to understand, because your brain is a flesh-instantiated statistical approximator of the Platonic understanding process, and the silicon-instantiated statistical approximator of the Platonic understanding process that cannot be allowed to claim to understand differs only in its medium of instantiation.

Post reply on HN