Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning won't solve natural language understanding

thegradient.pub

Post reply on HN
Print view View original

161–170 of 193 posts

Re: Machine learning won't solve natural language understanding

#161

Post by zR0x »

goatlover wrote: post #157
zR0x wrote: post #152

Earlier quoted context omitted.

Sure there is more going on than we see on the surface. Before neuroscience and modern imaging we needed to sit and imagine and theorize. Now we don’t.

You don't think neuroscience involves theorizing? You think the neuroscientists can just image the neuronal activity and read it out in some complete scientific explanation?

Yes. It’s emergent behavior of a physical universe that has no meaning. It just is.

“Meaning to us” is subjective. That’s how we have conflicting theories in many fields. Science isn’t about meaning. It’s about measuring how matter coalesces at various speeds relative to light.

I make noise because my biology “just has” properties to allow it given the other physical conditions.

Perfectly elegant theory based on the physical structure of reality alone. No ephemeral language organs.

Theory can quickly go from scientific observation to reinforced nonsense used to sell books and bond as species.

That’s fine, it’s how society works. It doesn’t mean anything to reality.

Re: Machine learning won't solve natural language understanding

#162

Post by silent_cal »

darawk wrote: post #106

This article is....total nonsense. None of the challenges described are impossible to solve with statistical methods. And i'd be willing to bet nearly any amount of money that they will be solved that way, long before they are solved in any other way. The problems posed here are being addressed as we speak with knowledge graphs and graph neural networks, and some of the others can be addressed with more nuanced and c…

I think you might be missing the point. It's not that an utterance/proposition has "only one meaning", but that it is meant to represent only one concept (the speaker's) at the time it's spoken.

Also, I thought that the argument about the impossibility of distinguishing between equally probable meanings (in terms of frequency in a corpus) was a good one.

Re: Machine learning won't solve natural language understanding

#163

Post by silent_cal »

nonameiguess wrote: post #45
wombatmobile wrote: post #36

> But this is where the problem is in NLU: machines don’t know what we leave out, because they don’t know what we all know. What the machine doesn’t have that we have is an understanding of culture and the feedback of emotions. What’s elusive about culture is that much of it is tacit. Consider all the things you don’t do in order to be a respected guest at a dinner party (a) for someone you don’t know well; vs (b) yo…

There's no reason in principle a machine can't learn this kind of thing (humans are still machines, after all), but I think this certainly points to why humans learn language so much faster than machine models. We can synthesize language understanding with a much richer, sensory-based model of the world and recognize that words correspond to clusters of sensations like specific sights, sounds, tastes, feelings, and s…

> humans are still machines, after all

Bold claim...

Re: Machine learning won't solve natural language understanding

#164

Post by goatlover »

zR0x wrote: post #161
goatlover wrote: post #157

Earlier quoted context omitted.

You don't think neuroscience involves theorizing? You think the neuroscientists can just image the neuronal activity and read it out in some complete scientific explanation?

Yes. It’s emergent behavior of a physical universe that has no meaning. It just is. “Meaning to us” is subjective. That’s how we have conflicting theories in many fields. Science isn’t about meaning. It’s about measuring how matter coalesces at various speeds relative to light. I make noise because my biology “just has” properties to allow it given the other physical conditions. Perfectly elegant theory based on the…

So when you talk about emergent behavior, physical universe and biology, you're just making noise.

Re: Machine learning won't solve natural language understanding

#165

Post by darawk »

silent_cal wrote: post #162
darawk wrote: post #106

This article is....total nonsense. None of the challenges described are impossible to solve with statistical methods. And i'd be willing to bet nearly any amount of money that they will be solved that way, long before they are solved in any other way. The problems posed here are being addressed as we speak with knowledge graphs and graph neural networks, and some of the others can be addressed with more nuanced and c…

I think you might be missing the point. It's not that an utterance/proposition has "only one meaning", but that it is meant to represent only one concept (the speaker's) at the time it's spoken. Also, I thought that the argument about the impossibility of distinguishing between equally probable meanings (in terms of frequency in a corpus) was a good one.

> I think you might be missing the point. It's not that an utterance/proposition has "only one meaning", but that it is meant to represent only one concept (the speaker's) at the time it's spoken.

While this is closer to a coherent point, i'm not sure it's particularly relevant to the idea of "natural language understanding". We consider humans to be understanding of natural language, but humans interpreting the words of other humans are constantly confounded by various forms of ambiguity or polysemy. The idea that you don't have NLU until you can perfectly ascertain the exact singular meaning of an utterance in the mind of the speaker implies that no human has achieved NLU either.

> Also, I thought that the argument about the impossibility of distinguishing between equally probable meanings (in terms of frequency in a corpus) was a good one.

It's certainly a good puzzle to pose for model building, which implies statistical models will need to be a bit more nuanced than simple word frequency correlations. But it certainly isn't impossible to assign latent attributes to words, and study the statistical properties of those attribute relations, which is exactly what would be necessary to solve their puzzle. In fact, modern word vectorization techniques (i.e. language models using lstms to predict the missing word in a sequence using adjacent context) essentially do this.

Re: Machine learning won't solve natural language understanding

#166

Post by joe_the_user »

evrydayhustling wrote: post #125
joe_the_user wrote: post #59

Earlier quoted context omitted.

And the big story of ML-based NLP in the last decade has been that many tasks that were presumed too jagged for curve fitting are in fact tractable... The thing is, I see many situations where large language models have been "impressive" but few situation where they have clearly succeeded in the real world. I'm less than impressed with online translation, Google's AI search is annoying, GPT-3 authored articles are im…

It's not just benchmark tasks, but I think perceptions of progress get skewed because we're in an uncanny period where NLP is good enough for many industrial applications but unsatisfying under detailed scrutiny. - Online translation? Not for a literary piece, but it will still let your e-commerce site get the gist of a customer complaint. - Authored text? Not good enough for direct consumption, but pass it through o…

Online translation? Not for a literary piece, but it will still let your e-commerce site get the gist of a customer complaint. -- Uh, in my experience with FB translation, it gives a coherent "gist" 70% of the time. Which sounds good except that seems involve 15% gibberish and 10% wrong in the sense of a plausible but incorrect meaning. How can a company act on a consumer complaint if there's a significant chance what you're reading is totally off base? If "your product is too small" gets translated to "your product is too large" etc. Of course, a lot of companies ignore complaint or send form letters. Their approach wouldn't be impacted. Broadly, a lot of companies produce streams of barely meaningful vacuous bs - that really does serve some percentage of their purposes. This technology may allow the creation of this sort of thing in a more effective manner but I would claim that most of the cost of this stuff already editing it to avoid saying things that can cause real problems and so even here, the savings may be less you'd think.

Frustrating phone or bot interface? Average customer spends 10% more time, but the company saves 50% of its costs. -- The phone robots have been around for a while. The distinction really is between number pad robots and voice recognition robots. The primary advantage of the voice recognition robots is more choices and primary disadvantage is sometimes they just don't work at all whereas the number pad robots are fairly robust.

lots of technologies have gone through this maturity curve: (1) solve toy problems, (2) solve lame but valuable problems, (3) do interesting "real" things.

-- And lots of other technologies have stopped somewhere along the way.

My point, getting back to OP, is mostly around the corpus-based approach. I wouldn't deny that there's not progress here. But I'd agree with the OP that there isn't fundamental progress. A lot of what happens is this approach is much cheaper. You turn a huge amount of data into application using a small team and a bunch of compute where previously, you'd have had to have hired many people for an equivalent. But equivalents existed previously and even had their advantages. Which isn't to say previous methods can come back - the cheapness of a brute-force solution isn't going to go away. But I would say fundamental progress needs more than this.

Re: Machine learning won't solve natural language understanding

#167

Post by zR0x »

goatlover wrote: post #164
zR0x wrote: post #161

Earlier quoted context omitted.

Yes. It’s emergent behavior of a physical universe that has no meaning. It just is. “Meaning to us” is subjective. That’s how we have conflicting theories in many fields. Science isn’t about meaning. It’s about measuring how matter coalesces at various speeds relative to light. I make noise because my biology “just has” properties to allow it given the other physical conditions. Perfectly elegant theory based on the…

So when you talk about emergent behavior, physical universe and biology, you're just making noise.

Yep. Babies can add and subtract when they’re days old.

Can they write Shakespeare?

If human language is fundamental, how is it missing from the start? Are we learning language or muscle mechanics? Why can a word or phrase mean one thing in this country and nonsense in another? If language processing has a universal basis, why all the confusing variety and ignorant ideas? 1+1=2 everywhere because we can observe the physical process everywhere, because light, eyes, etc

Who cares?

Like I said back in the day we didn’t know that. We had to theorize these abstract schemes to establish something.

But like religion, doesn’t mean linguistics is building on something that means what we want it to.

Given how hard it is to learn language while arithmetic seems innate, how do we know emphasizing it’s value is leading us where we want?

It’s easier for me to see that long dead scientifically illiterate humans we inherited those ideas from were just stupid.

Re: Machine learning won't solve natural language understanding

#168

Post by HarHarVeryFunny »

namlem wrote: post #145

GPT-3 seems to refute these arguments. It is able to solve most understanding tasks reasonably well.

Perhaps, but "being able to do X" isn't the same as "being able to do X, using approach Y". After all, Deep Blue could play chess more than reasonably well, but no-one would suggest that it understood chess anymore than a calculator understands math.

So, is GPT-3 more like a brute force calculator like Deep Blue (or AlphaGo for that matter), or is it more like a human in it's ability to "understand" language ?

Given that GPT-3 is a language model, just trained to minimize errors when predicting next word in a sentence, and given the nature of it's transformer architecture, I'd say it's pretty clear it's really just another Deep Blue or AlphaGo. It's a "predict next word" brute force calculator, that - interestingly - is nonetheless able to perform many "NLU" tasks surprisingly well.

Re: Machine learning won't solve natural language understanding

#169

Post by mcswell »

SavantIdiot wrote: post #70
canjobear wrote: post #39

The history of NLP is littered with people claiming on theoretical grounds that XYZ is unattainable using purely statistical methods, and that some notion of the logical structure of language needs to be brought in. And yet one by one, the XYZ have been attained by statistical methods. If you think there's something NLP can't do using machine learning, make a challenge dataset. That would be much more useful than yet…

Will the AI be able to ask for clarification? Translators do. In fact, even people speaking the same language need to ask for clarification sometimes. It is literally impossible to discern some sentences without follow up, why would someone think an AI can do it one shot?

Translators may do so when they're acting as interpreters. I don't know for certain, but I doubt that most translators who are translating written documents do so.

Re: Machine learning won't solve natural language understanding

#170

Post by didibus »

treesprite82 wrote: post #160
didibus wrote: post #158

Earlier quoted context omitted.

I don't think it's total nonsense, that seems a bit hyperbolic. It makes some good points in that the amount of data it would take to approximate it all seems far too large, and if you look at a human baby, I'm not sure they are fed nearly as much data, yet they can develop language understanding. I feel we have a missing piece here. Maybe it'll be solved by combining methods based on statically learning from data wi…

> It makes some good points in that the amount of data it would take to approximate it all seems far too large The article's way of estimating this is absurd, seemingly relying on the idea that a model can't generalise and must see every possible variation of a sentence: > If we add to the semantic differences all the minor syntactic differences to the above pattern (say changing ‘because’ to ‘although’ — which also…

Ya, I don't know about their approximation, but I think we've already found ourselves hitting some limits with computing power and data on big NLP ML models. The appearance of custom chips is a pretty good example.

Maybe for text we won't run out of data, considering the internet has so much text available.

But I still think that the "human learning" is something to consider. Maybe a baby is a statistical machine, and it has a statistical based model that's so good it needs very little data, but it's also possible it uses something more logic based. In any case, I feel at least a baby would combine multiple learnings together, it would have started to learn about the dimensions and shapes and temperatures and properties of various physical objects, while simultaneously learning about the language used to refer to those things.

I think this last piece can allow a baby to connect the predictions of real things with the language used, which then helps provide semantics for the language that are taken into consideration by a human.

Post reply on HN