Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning won't solve natural language understanding

thegradient.pub

Post reply on HN
Print view View original

31–40 of 193 posts

Re: Machine learning won't solve natural language understanding

#31

Post by drdeca »

foldr wrote: post #19
drdeca wrote: post #12

Going through this, it has the old "A full understanding of an utterance or a question requires understanding the one and only one thought that a speaker is trying to convey. " claim, which continues to not make any sense, because obviously people don't do that; as much as I would like to be understood in precisely the way I mean, down to the most subtle nuance/shade of meaning and connotation, at least much of the t…

> Also, claiming that natural language is infinite, if taken literally, would imply a large and contrary to the common consensus claim about physics, contradicting the Bekenstein bound and all that. Natural language is infinite in the pretty straightforward sense that, say, chess is infinite (there is an infinite number of valid chess games - if you ignore arbitrary restrictions such as the 50 move rule). This of cou…

Natural language as understandable by humans, is finite, because humans are finite, at least in this life.

There are only finitely many distinct utterances that a person could possibly hear that have length less than one lifetime.

Any utterance which takes longer than a lifetime to make/hear, is not really natural language, so much as an extrapolation of what natural language would/might be if people had unlimited lifespans, memory, and attention-spans.

How would I represent intensions in a neural network? Well, you can encode text with a sequence of 1-hot vectors. (Is this trivial? Yes. Still counts.) If you can encode intensions on a computer, you can encode it as text. If you can encode it as text, you can encode it as a vector.

Do I think that (a sequence of 1-hot vectors) is the best way to do it? Not really, no. I'd need a bit more detail on what is meant to be represented in order to give a better shot at describing what I think could be a good approach for encoding it.

But also, I don't think the burden of proof is on me here. The author claimed that it is impossible, I said I don't see any justification for that claim.

Personally, I'm not entirely sure what they are saying is impossible. Do they have a particular task in mind?

____

Also, the set of possible states in a game of chess is finite, and a complete understanding of how to play chess optimally would, as such, also be finite. The fact that you can repeat some subset of states in a bunch of orders for an arbitrary amount of time, before you stop messing around and actually finishing the game, doesn't grant chess infinite complexity in any meaningful way.

A finite state machine can recognize a language which has infinitely many strings in it, yes. This does not mean that a RNN can't emulate such an FSM.

Re: Machine learning won't solve natural language understanding

#32

Post by kens »

The article seems to claim that machine learning won't work for natural language understanding because ML uses the Probably Approximately Correct (PAC) paradigm but language understanding must be exactly correct. As far as I can tell, that's nonsense, completely misunderstanding the PAC framework.

Re: Machine learning won't solve natural language understanding

#34

Post by Isinlor »

I will refute his thesis argument by argument.

1.

> Do we have a retired BBC reporter that was based in an East European country during the Cold War? > (...) None of the above challenging semantic understanding functions can be ‘approximately’ or ‘probably’ correct – but absolutely correct.

There already exists systems that do text to sql translation. The question author uses as an example is actually quite easy, and not where modern text to sql systems tend to fail. The "absolutely correct" part is clearly false. Both humans and ML systems will make mistakes and have to deal with ambiguity. As a software developer working with nontechnical stakeholders, I'm 100% convinced that people routinely compose sentences they do not understand in any absolute sense.

2.

Good way to check if your proof is correct, is to check if it leads to absurd conclusions. His proof related to learnability (ML) and compressibility (COMP) would seem to indicate that either people can not learn or they do not understand language. Absurd.

3.

> In ML/Data-driven approaches there is no type hierarchy where we can make generalized statements about a ‘bag’, a ‘suitcase’, a ‘briefcase’ etc. where all are considered subtypes of the general type ‘container’. Thus, each one of the above, in a purely data-driven paradigm, are different and must be ‘seen’ separately in the data.

This is objectively false. Even simple context free embeddings like Word2Vec will capture some relationships between a ‘container’, a ‘bag’, a ‘suitcase’, a ‘briefcase’. Based on that alone the rest of that argument falls apart. But large language models go beyond context free embeddings.

4.

The last argument is relatively good one. It's hard to extract all attributes of human concepts from text alone. This is where large language models fail miserably. We need to provide models access to different modalities, but not only. The models can not be embedded in a static world of recordings in order to create rich understanding of things like agent or agency etc.

This is indeed missing, but at the same time this is being actively worked on.

So, that's it. Author makes 3 either false or absurd arguments. And one that is good, but wholly unconvincing as to impossibility of ML solving natural language understanding.

Re: Machine learning won't solve natural language understanding

#35

Post by nonameiguess »

lalaithion wrote: post #5

> In other words, we must get, from a multitude of possible interpretations of the above question, the one and only one meaning that, according to our commonsense knowledge of the world, is the one thought behind the question some speaker intended to ask. But, can humans do this? I think not; I still disagree with the author about what "Do we have a retired BBC reporter that was based in an East European country duri…

I don't think you interpreted the sentence in question differently, just the nature of how it is being translated into a query. They're talking specifically of how to translate into a query over a quantified relational algebra. Relational algebra always operates on and return sets. I'm pretty sure the author here is still looking for a boolean, but you can't ask a relational query engine does a result exist? You can only get back sets. The further question of whether or not the set is nonempty is an extra feature probably built into most real-world implementations of query engines, but it's not part of relational algebra.

The actual problem they're identifying here is that of deciding how or whether to turn 'a' into a quantifier. Do we have 'any' would be less ambiguous, definitely an existential quantifier. Do we have 'a' depends on context, as it could mean one specific person and not be a quantifier at all, but I actually still disagree with the author here. I think they're right that English-speaking humans will unambiguously understand 'a' in this context to be a quantifier, but given that is possible at all, I don't see why a large language model can't. If it was actually being used to specify one person, the additional context would be in the sentence itself, as in 'do we have a reporter named X' or something like that. Given there is no additional qualification in the sentence, I'm pretty sure existing large language models can easily figure out what it means, and the only case in which they'd be wrong is if there were additional context outside of the sentence, but a human would be wrong too if all they heard was the sentence.

Re: Machine learning won't solve natural language understanding

#36

Post by wombatmobile »

> But this is where the problem is in NLU: machines don’t know what we leave out, because they don’t know what we all know.

What the machine doesn’t have that we have is an understanding of culture and the feedback of emotions.

What’s elusive about culture is that much of it is tacit.

Consider all the things you don’t do in order to be a respected guest at a dinner party (a) for someone you don’t know well; vs (b) your best friend’s mother; vs (c) your new boss after the takeover. Most of the decisions you make are the result of tacit feedback (or lack thereof) that a machine can’t access.

Consider these lines from an Eagles song:

City girls just seem to find out early

How to open doors with just a smile

How’s a machine to learn that knowledge, when the substantive information never passes through a keyboard or a camera, and only exists in response to biological stimuli?

Re: Machine learning won't solve natural language understanding

#37

Post by foldr »

drdeca wrote: post #31
foldr wrote: post #19

Earlier quoted context omitted.

> Also, claiming that natural language is infinite, if taken literally, would imply a large and contrary to the common consensus claim about physics, contradicting the Bekenstein bound and all that. Natural language is infinite in the pretty straightforward sense that, say, chess is infinite (there is an infinite number of valid chess games - if you ignore arbitrary restrictions such as the 50 move rule). This of cou…

Natural language as understandable by humans, is finite, because humans are finite, at least in this life. There are only finitely many distinct utterances that a person could possibly hear that have length less than one lifetime. Any utterance which takes longer than a lifetime to make/hear, is not really natural language, so much as an extrapolation of what natural language would/might be if people had unlimited li…

Sure, but this is missing the point. There’s also a finite number of C programs that can actually exist in the physical universe, but that’s an arbitrary limit, not part of the definition of C. Similarly, there’s no non-arbitrary limit on, say, the length of an English sentence.

All of this ‘debate’ about the infinity of language just reduces to a misunderstanding of what people are saying, as far as I can see. No-one thinks that more than a finite number of English sentences will ever be uttered; conversely, no-one thinks that we will ever discover such a thing as a complete list of all English sentences (since trivially the conjunction of all these sentences would be a new sentence not on the original list).

Note that if you view English as a regular tree language, your point about chess also applies to English. You don’t need to remember all of the preceding state, just the congruence class. No-one is saying that English is “infinitely complex”. The grammar of English is finite, but there is no limit on the number of sentences that it can assign a structure to.

Of course you can encode intensions using numbers in the broad sense that you can encode pretty much anything using numbers.

Re: Machine learning won't solve natural language understanding

#38

Post by h0l0cube »

drdeca wrote: post #21
emodendroket wrote: post #14

Earlier quoted context omitted.

That's a different problem, isn't it? That's more about transcription -- getting the speech into words -- than about what he's talking about, making sense of the words once you have them.

The two tasks are interconnected. The reasoning flows both ways.

Noise and lack of specificity are completely different problems, and the article concerns itself with the latter

Re: Machine learning won't solve natural language understanding

#39

Post by canjobear »

The history of NLP is littered with people claiming on theoretical grounds that XYZ is unattainable using purely statistical methods, and that some notion of the logical structure of language needs to be brought in. And yet one by one, the XYZ have been attained by statistical methods.

If you think there's something NLP can't do using machine learning, make a challenge dataset. That would be much more useful than yet another theoretical argument.

Post reply on HN