Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI’s Language Problem

technologyreview.com

Post reply on HN
Print view View original

21–30 of 244 posts

Re: AI’s Language Problem

#21

Post by dmreedy »

flaviuspopan wrote: post #3

Is there any hope that if chat bots do actually become widespread in usage that eventually we'll be able to aggregate their collective knowledge similar to reinforcement learning for a single system? That seems like the only likely way we'll ever be able to train AI in something as complex as language.

It depends on what the nature of language is. If it is purely a tool of consciousness, a separate module in mind like basic image recognition seems to be, then the picture you're painting may be possible. It may be that the language signal is strong enough to pick up just by looking at orders of magnitude more examples than we can right now. That we can crack the encryption, so to speak.

However, if language is more integral than that. If language is more a facet of intelligence than a building block. If language is the structure of sentience, rather than something sentience leverages, then no, all the chat bots in the world won't help. We need something that can integrate more than just plain text embeddings of incredibly intricate and complex structures. We can't crack this code without a key.

My guess is that this latter scenario is the likely case.

Re: AI’s Language Problem

#22

Post by Severian »

I have to agree to an extent with researchers Tenenbaum and Li. It seems to me that the only way AI is going to learn language is to have some worldly experience to link words to their ultimate semantics.

I don't think AI will be able to fully grasp the intricacies of human language until it has "lived" long enough to form the links between ideas and experiences. Mainly in the physical realm, as obviously a lot of our human development is shaped by our environments. They will need eyes, ears, and maybe noses. We should also consider giving them subconscious or instinctive reactions to certain stimuli. An AI wouldn't immediately know that, for example, rotten meat is bad to humans because it lacks a nose to send a signal of danger and disgust.

We should also consider the idea to communicate with AI in regular face-to-face speech. Talking is not the same as writing, and conveys a lot of information beyond just the words.

Re: AI’s Language Problem

#23

Post by adwn »

Practicality wrote: post #4

If we use animals as a reference, I would say that consciousness is more fundamental than language, so most likely we need that in place before we can get AI to be able to effectively understand language.

Do you mean "consciousness" as being aware of one's own existence and relative position in a larger reality, or as having subjective experiences (qualia, feelings)?

Re: AI’s Language Problem

#24

Post by bbctol »

EGreg wrote: post #15
ideonexus wrote: post #11

I think about the AI language problem a lot while raising my kids. The article notes the word "forever" and how an AI must distinguish the literal from the figurative meaning of the word in context. My five-year-old still doesn't grasp the literal meaning of this word as "never-ending." To him, "forever" is simply a very very long time. He has the same problem with the concept of "infinity," where the word means both…

Maybe this is good. After all, mathematical abstractions may cause more philosophical problems than they solve. What if there is nothing infinite in this world? Having a firm grasp of reality before venturing into hypotheticals can be good.

But what is language if not a tool for manipulating hypotheticals? If your language can only describe what you know to be possible, it can only describe things you've already seen, and the required dataset in your memory to have a conversation or provide useful information is way too large. Abstraction is the very problem of language that AIs are trying to solve.

Re: AI’s Language Problem

#25

Post by wildmusings »

Consciousness is the secret sauce. Consciousness as in "subjective experience", what separates us from philosophical zombies, the sensation of the color blue, of middle-c. Our bodies evolved and kept this extremely rich phenomenon for a reason; it is extraordinarily unlikely for it to have arisen and remained through genetic drift alone.

My theory, and I'd love to find someone offering a similar and more fleshed-out hypothesis, is that conscious experience serves as a universal data type. It can encode and play-back any type of knowledge and memory, and relationships among them, from the color of the teacher's shirt that time you broke your bone in 3rd grade, to the formula for electron energy in quantum mechanics.

Unfortunately, the word consciousness is almost forbidden in most scientific circles. The dominant view is that there is no Hard Problem of Consciousness and that any discussion of it is quackery, or at least "not science". This taboo is holding us back.

Re: AI’s Language Problem

#26

Post by dmreedy »

brianchu wrote: post #8

Deep learning has succeeded tremendously with perception in domains that tolerate lots of noise (audio/visual). Will those successes continue with perception in domains that are not noisy (language) and inference/control , which the article touches on? I think it really is unclear whether those challenges will require fundamental developments or just more years of incremental improvement. If fundamental developments…

I think you're dead on. And I'm nervous about a coming winter, because of disappointment in all the wolf-crying we're doing about how good at Natural Language we're getting. When we've barely scratched the surface. This latest bot fad worries me.

A further comment on deep methods being state of the art currently:

I wonder how well these tasks really measure progress in natural language understanding (I really don't like isolating that term as some distinct subdiscipline of broader AI goals, but so be it). Some of Chris Manning's students[1] have at least started down the path of examining some of these new-traditional tasks in language, and found that perhaps they are not so hard as they claim to be.

---

[1] A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task. Chen, Bolton & Manning [https://arxiv.org/abs/1606.02858]

Re: AI’s Language Problem

#27

Post by Retric »

brianchu wrote: post #8

Deep learning has succeeded tremendously with perception in domains that tolerate lots of noise (audio/visual). Will those successes continue with perception in domains that are not noisy (language) and inference/control , which the article touches on? I think it really is unclear whether those challenges will require fundamental developments or just more years of incremental improvement. If fundamental developments…

Language is noisy. People often say things that have little to do with what they mean and context is really important.

EX: "How long do stars last?" Means something very different in a science class than a tabloid headline. Is that tabloid talking divorce or obscurity? Notice how three sentences in I am clarifying last.

Re: AI’s Language Problem

#28

Post by brianchu »

dmreedy wrote: post #26
brianchu wrote: post #8

Deep learning has succeeded tremendously with perception in domains that tolerate lots of noise (audio/visual). Will those successes continue with perception in domains that are not noisy (language) and inference/control , which the article touches on? I think it really is unclear whether those challenges will require fundamental developments or just more years of incremental improvement. If fundamental developments…

I think you're dead on. And I'm nervous about a coming winter, because of disappointment in all the wolf-crying we're doing about how good at Natural Language we're getting. When we've barely scratched the surface. This latest bot fad worries me. A further comment on deep methods being state of the art currently: I wonder how well these tasks really measure progress in natural language understanding (I really don't l…

I don't think there will be a winter. There are enough successes in computer vision.

Re: AI’s Language Problem

#29

Post by brianchu »

Retric wrote: post #27
brianchu wrote: post #8

Deep learning has succeeded tremendously with perception in domains that tolerate lots of noise (audio/visual). Will those successes continue with perception in domains that are not noisy (language) and inference/control , which the article touches on? I think it really is unclear whether those challenges will require fundamental developments or just more years of incremental improvement. If fundamental developments…

Language is noisy. People often say things that have little to do with what they mean and context is really important. EX: "How long do stars last?" Means something very different in a science class than a tabloid headline. Is that tabloid talking divorce or obscurity? Notice how three sentences in I am clarifying last .

Sorry, I've edited my original comment to be clearer. What I really meant is that there is wide tolerance of noise in those domains. "How long does stars last" has a completely different meaning than "How long do stars last" - not tolerant of noise.

Re: AI’s Language Problem

#30

Post by mark_l_watson »

My intuition is that the language problem will be solved, but that the solution will likely be a hybrid symbolic and deep learning system. BTW, I have been working (some of the time) on both symbolic NLP and neural networks/machine learning since the 1980s: right now is the most exciting time in the field of AI because progress is rapid and accelerating.
Post reply on HN