AI’s Language Problem
technologyreview.com
AI’s Language Problem
1–10 of 244 posts
Re: AI’s Language Problem
#2My belief is that "Deep Network" systems will fail to produce commercially useful results on language just as symbolic systems failed in the face of visual recognition.
Re: AI’s Language Problem
#3Is there any hope that if chat bots do actually become widespread in usage that eventually we'll be able to aggregate their collective knowledge similar to reinforcement learning for a single system? That seems like the only likely way we'll ever be able to train AI in something as complex as language.
Re: AI’s Language Problem
#4If we use animals as a reference, I would say that consciousness is more fundamental than language, so most likely we need that in place before we can get AI to be able to effectively understand language.
Re: AI’s Language Problem
#5It seems to me that a full mastery of language requires a grasp of semantics, that is the ability to understand what a sentence means. I doubt it's possible to do that without having basic common sense along with an overall representation of the world, and that looks very close to strong AI, imho.
So I'm not surprised computers keep on struggling with language applications. Once they succeed strong AI will not be much further away.
Re: AI’s Language Problem
#6This is also why AI won't overthrow us and become our masters. They don't even know what it means to be a master.
EDIT: This is the most volatile comment I've ever posted. It has been going +2, -2, +2, -2 for the last 35 minutes. People seem to love it or hate.
Re: AI’s Language Problem
#7These guys might disagree. Sometimes they make sense which is interesting http://sumve.com/ai-chatbots/relationships/relationship-bots...
Re: AI’s Language Problem
#8Deep learning has succeeded tremendously with perception in domains that tolerate lots of noise (audio/visual). Will those successes continue with perception in domains that are not noisy (language) and inference/control, which the article touches on? I think it really is unclear whether those challenges will require fundamental developments or just more years of incremental improvement. If fundamental developments are needed, then the timeline for progress - which everyone in tech seems to be interested in - becomes much more indeterminate.
If you think about audio/visual data, deep nets make sense: if you tweak a few pixel values in an image, or if you shift every pixel value by some amount, the image will still retain basically the same information. In this context, linearity (weighting values and summing them up) make sense. It's not clear whether this makes sense in language. On the other hand, deep methods are state of the art on most NLP tasks, but their improvement over other methods isn't the huge gap as in computer vision. And while we know there are tight similarities between lower-level visual features in deep nets and the initial layers of the visual cortex, the justification for deep learning in NLP is simpler and less specific: what I see is the fact that networks have huge capacity to fit to data and are deep (rely on a hierarchy of features). My guess is we may need a fundamental breakthrough in a newfangled hierarchical learning system that is better suited for language to “solve” NLP.
I think there are similar limitations with control and inference. When it comes to AlphaGo the deep learning component is responsible for estimating the value of the game state; the planning component is done with older methods. This is much more speculative, but when it comes to the work on Atari games, for example, I suspect that most of what is being learned (and solved) is perception of useful features from the raw game images. I wonder whether the features for deducing game state score are actually complex.
I think what I'm trying to say is that when we look at the success of deep learning, we have to separate out what part of that is due to the fact that deep learning is the go-to blackbox classifier, and what part of this is due to the systems we use actually being a good model for the problem. If the model isn't good, does that model merely need to be tweaked from what we currently use, or does the model have to completely change?
Re: AI’s Language Problem
#9This is a long metaphor but work with me here...
There are many types of application programmers, but there are 2 types in particular that are interesting. One of them is the purely technology-driven developer. He uses all the new tools, he's read Knuth's books a hundred times, he knows how to build elegant systems. However, he only takes enough interest in the business as is necessary to know what to build. At the end of the day, he'll build the most elegant beautiful system that almost never accomplishes the business goal. He knows how to describe a problem, but he doesn't really know the problem.
The second likes programming, he finds technology fun, but he is really driven by trying to understand the full context of the business. Writing software is a means to see an impact on people. He's driven by seeing a business problem solved. I've only met 2 people in my career who are ACTUALLY like this, they're rare... which is maybe a good thing because they write shit code.
A good engineering team tries to get both of these guys, you have the tech guy making sure your platform is maintainable, and you have the business driven guy who makes sure it's useful. One guy understands the structure of the tool, the other understands the structure of the world the tool is in.
A language is a tool, it can be elegant, it can be beautiful, it can be technically perfect.. and just like poetry, it can have a very little practical utilitarian purpose. When I look at how we're using ANN's to develop language today, this is how it feels to me. We're spending so much time trying to figure out how to get a computer to build the most technically perfect sentence, we're missing the maybe more interesting problem of trying to get a computer to understand the world. My son right now isn't old enough to conjugate a sentence, but he understands what certain things in the world do. He clearly understands that cars move things, he understands you can use the hose to get things wet. He's not that old, but he's developing a mental model of the world. He just doesn't know how to describe it yet.
To me, having a computer look at a crane, and then print the word "crane" is interesting, but even more interesting is if you could give it 3 pictures (a crane, a building, and a pile of rubble) and teach it how 1+1=2.
Re: AI’s Language Problem
#10Is there any hope that if chat bots do actually become widespread in usage that eventually we'll be able to aggregate their collective knowledge similar to reinforcement learning for a single system? That seems like the only likely way we'll ever be able to train AI in something as complex as language.
The primary issue is that language doesn't have as well of a defined "success" metric. So more data doesn't necessarily make better language.
Without a human analyzing the transcripts it's very difficult for the chat bots to know which inputs it's receiving are "good" or "better."
Even the idea of good language is subjective. We all know there is such a thing, but nearly everyone has different ideas of what this is.