Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI’s Language Problem

technologyreview.com

Post reply on HN
Print view View original

31–40 of 244 posts

Re: AI’s Language Problem

#31

Post by komali2 »

ideonexus wrote: post #11

I think about the AI language problem a lot while raising my kids. The article notes the word "forever" and how an AI must distinguish the literal from the figurative meaning of the word in context. My five-year-old still doesn't grasp the literal meaning of this word as "never-ending." To him, "forever" is simply a very very long time. He has the same problem with the concept of "infinity," where the word means both…

Kids are an awesome way to think about these problems. I did a teaching abroad stint once and it was incredible to witness kindergartners wrastle with this new language I was introducing to them, adopt it while they're still learning their native one. It got even cooler when there'd be a half native kid who already spoke English as well as Chinese.

I wonder if there's AI focused research that analyses how children learn language as an aspect of their research? Especially kids learning a "new" language.

Re: AI’s Language Problem

#32

Post by lottin »

Practicality wrote: post #4

If we use animals as a reference, I would say that consciousness is more fundamental than language, so most likely we need that in place before we can get AI to be able to effectively understand language.

I agree. It seems to me that before language can develop you must have ideas, such as notions of space and time and existence, and in order to have ideas you probably need some kind of sensory apparatus that tells you things about yourself and about the environment you are in. But since we don't really understand how these things work, I doubt that we'll be able to recreate anything like it artificially.

Re: AI’s Language Problem

#33

Post by dmreedy »

Retric wrote: post #27
brianchu wrote: post #8

Deep learning has succeeded tremendously with perception in domains that tolerate lots of noise (audio/visual). Will those successes continue with perception in domains that are not noisy (language) and inference/control , which the article touches on? I think it really is unclear whether those challenges will require fundamental developments or just more years of incremental improvement. If fundamental developments…

Language is noisy. People often say things that have little to do with what they mean and context is really important. EX: "How long do stars last?" Means something very different in a science class than a tabloid headline. Is that tabloid talking divorce or obscurity? Notice how three sentences in I am clarifying last .

Yep. The problem is that it's _so_ noisy, that the encryption, as it were, might be too strong to crack with statistical methods. You might need the key; i.e., something like a human brain.

EDIT: a combination of noise, I should say, and paucity of information.

Re: AI’s Language Problem

#34

Post by dmreedy »

brianchu wrote: post #28
dmreedy wrote: post #26

Earlier quoted context omitted.

I think you're dead on. And I'm nervous about a coming winter, because of disappointment in all the wolf-crying we're doing about how good at Natural Language we're getting. When we've barely scratched the surface. This latest bot fad worries me. A further comment on deep methods being state of the art currently: I wonder how well these tasks really measure progress in natural language understanding (I really don't l…

I don't think there will be a winter. There are enough successes in computer vision.

Yeah, absolutely, and those successes are not going anywhere. But as solutions to those problems become more and more rote, funding will still be needed for the bigger problem, which continues to fail to deliver on its promises.

Re: AI’s Language Problem

#35

Post by TeMPOraL »

inputcoffee wrote: post #18
philh wrote: post #17

Earlier quoted context omitted.

Nor does a meteor know what it means to be an extinction event.

True but a meteor doesn't have to. As it has been written of in our sci-fi, the robot uprising specifically requires the machines to understand and, most of all, care, about dominance. If you're talking about just being victims of machine logic, we've been suffering that since the invention of the traffic light traffic jam.

That's because fiction is fiction, especially one created for the masses must be written in terms masses can relate to.

As for reality, creating an AI that will actively hate us is a feat of about the same difficulty as creating an AI that would love us. Those are two opposite points of a tiny island called "has more-less human mind" that floats on a vast ocean named "we all die". The biggest challenge of surviving superhuman AIs is locating that island.

As the words of wisdom say, "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else."

Re: AI’s Language Problem

#36

Post by Practicality »

adwn wrote: post #23
Practicality wrote: post #4

If we use animals as a reference, I would say that consciousness is more fundamental than language, so most likely we need that in place before we can get AI to be able to effectively understand language.

Do you mean "consciousness" as being aware of one's own existence and relative position in a larger reality, or as having subjective experiences (qualia, feelings)?

It's a state of being able to say "I am doing X, I am doing Y. I am thinking Z. (but not necessarily in words of course)" It's also a creation of an executive process that is separate from the analytic processes. The executive process can be aware of the analytic processes (Like we can know about our heart beating, but it's a separate process)

Subjective feelings are not necessarily part of that.

This article on how consciousness evolved does a good job of explaining how it works (finally), and I think it's something we could emulate.

http://www.theatlantic.com/science/archive/2016/06/how-consc...

Re: AI’s Language Problem

#37

Post by king_magic »

wildmusings wrote: post #25

Consciousness is the secret sauce. Consciousness as in "subjective experience", what separates us from philosophical zombies, the sensation of the color blue, of middle-c. Our bodies evolved and kept this extremely rich phenomenon for a reason; it is extraordinarily unlikely for it to have arisen and remained through genetic drift alone. My theory, and I'd love to find someone offering a similar and more fleshed-out…

I've had somewhat similar thoughts, and I am entirely unqualified (and highly likely not the first) to put forth the idea that consciousness's killer app is the ability to rapidly assembly abstract models of experiences (present from current sensory input, past from short/long-term memory or future from mental simulation) and be able to query/manipulate those models, and my (admittedly potentially naive) suspicion is that whatever does "that" is the machinery behind consciousness.

This is also why I think that deep learning / neural networks are only going to take us so far. I think there is more to the story of how the brain works than only neural networks that make predictions, and frankly I do not think that any system that does not at least attempt to do "that" (simulating consciousness's model building/manipulation feature) will have much better luck at language processing/understanding.

Re: AI’s Language Problem

#38

Post by KasianFranks »

The focus on vector space as mentioned in the article "words can be represented as mathematical vectors, allowing similarities between related words to be calculated. For example, “boat” and “water” are close in vector space even though they look very different. Researchers at the University of Montreal, led by Yoshua Bengio, and another group at Google, have used this insight to build networks in which each word in a sentence can be used to construct a more complex representation—something that Geoffrey Hinton, a professor at the University of Toronto and a prominent deep-learning researcher who works part-time at Google, calls a “thought vector.”"

Is to me, the most significant way in which we can mimic the way the human cognitive process develops associations between things. Auto-association is key here.

In addition, understanding how to calculate similarities between vectors is also important.

Re: AI’s Language Problem

#39

Post by jd20 »

Solving language for computers seems much like climbing a series of mountains, where each time you surmount one, you realize the next is even higher :) Thanks to deep learning, machines have made rapid gains in speech recognition, as well as improving semantic mapping (a la word2vec and other word embedding approaches).

But once you have a system with human-level speech recognition and semantic mapping, where do we go? The ability to have a meaningful dialogue with a machine seems very difficult to model as a machine learning problem (what constitutes ground truth? What's the reward function look like?), and also has to deal with many unknowns. For example, ask a smart assistant like Alexa or Siri about functionality it wasn't programmed with, and you get a terse "Sorry, can't help you with that." But ask a child, and you prompt a question-answer dialogue (i.e. learning) or perhaps feigned understanding. My toddler son is an expert at giving me the answer he thinks I want to hear, even when he has no idea what I'm talking about :) There are certainly many new problems which we can begin to think about tackling, but certainly no sign IMO that we're running out of applications for deep learning in the field of language.

Re: AI’s Language Problem

#40

Post by jerf »

brianchu wrote: post #29
Retric wrote: post #27

Earlier quoted context omitted.

Language is noisy. People often say things that have little to do with what they mean and context is really important. EX: "How long do stars last?" Means something very different in a science class than a tabloid headline. Is that tabloid talking divorce or obscurity? Notice how three sentences in I am clarifying last .

Sorry, I've edited my original comment to be clearer. What I really meant is that there is wide tolerance of noise in those domains. "How long does stars last" has a completely different meaning than "How long do stars last" - not tolerant of noise.

I'd suggest "How long do these stars last?" and "How long do these stairs last?" might be a better example. Human language has more redundancy than computer languages and in a real context it would probably still be clear what was meant even if the wrong word was used, but it's still a much spikier landscape with regard to small changes than images are.
Post reply on HN