The problem with deep learning and language understanding is that the task is ill-defined end-to-end. For speech, image understanding, and translation, you can come up with large datasets of x->y and have deep learning learn a complex function to approximate the mapping. We don't have that luxury in language understanding, at least not yet.
AI’s Language Problem
91–100 of 244 posts
Re: AI’s Language Problem
#92No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…
Why not simply simulate the body (or indeed multiple bodies) in a virtual world?
Easier to build, parallelize, extend, maintain. Possibly somewhat safer too.
Re: AI’s Language Problem
#93I think about the AI language problem a lot while raising my kids. The article notes the word "forever" and how an AI must distinguish the literal from the figurative meaning of the word in context. My five-year-old still doesn't grasp the literal meaning of this word as "never-ending." To him, "forever" is simply a very very long time. He has the same problem with the concept of "infinity," where the word means both…
That reminded my of this blog article by Scott Aaronson: http://www.scottaaronson.com/writings/bignumbers.html
Re: AI’s Language Problem
#94Earlier quoted context omitted.
I haven't heard of an AI system understanding that yet but it's a fairly common concept in everyday life for instance dogs tend to have masters. Anyone trying to make an AI understand everyday life or literature is going to have to deal with that.
I think the article speaks to this very point. The people "trying to make an AI understand" are not making it "understand" in the sense that you and I use most commonly use the term.
You're exactly right. If you look closely, present day A.I is modeled after the cortex. It's akin to cutting out the neo-cortex, wiring in i/o hooks, and reprogramming it to one's needs (weak A.I).
The cortical regions are good at creating hierarchical feature maps and we had a bunch of search algorithms laying around in the parts bin. Presto : Present day A.I.
This approach meshes well with 'big-data' companies in possession of large compute stacks and data-sets. So, its the direction things went.
A perfect opportunity for disruption.. The current wave is on borrowed time.
Re: AI’s Language Problem
#95No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…
Why not simply simulate the body (or indeed multiple bodies) in a virtual world? Easier to build, parallelize, extend, maintain. Possibly somewhat safer too.
That works, if your virtual world is good.
But that's a big if. You've just taken one really hard problem (learning about the world) and turned it into an even harder problem (simulating the world).
Re: AI’s Language Problem
#96Earlier quoted context omitted.
Yep. The problem is that it's _so_ noisy, that the encryption, as it were, might be too strong to crack with statistical methods. You might need the key; i.e., something like a human brain. EDIT: a combination of noise, I should say, and paucity of information.
No, you only need context. Context in the form of knowledge about the place, company and history that the statement is spoken in. Wikipedia will serve well for a lot of that.
Representation of relationships without representation of qualia gives you brittle nonsense - a content-free wireframe of word distributions.
For human-level NLP, you need to model the mechanism by which the relationship network is generated, and ground it in a set of experiences - or some digital analogue of experiences.
Naive statistical methods are not a good way to approach that problem.
So no, Wikipedia will not provide enough context, for all kinds of reasons - not least of which is the fact that human communications include multiple layers of meaning, some of which are contradictory, while others are metaphorical, and all of the above can rely on unstated implication.
Vector arithmetic is not a useful model for that level of verbal reasoning.
Re: AI’s Language Problem
#97No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…
It's probably a mistake to assume that just because it's the way we do it that it has to be the way machines do it. Although that's usually the initial assumption. In the early days of flight most attempts were based on birds, similarly submersible vehicles were based on fish. We know now it's better to use propellers. It could be we just haven't found what is analogous to a propeller for the AI world.
Re: AI’s Language Problem
#98If we use animals as a reference, I would say that consciousness is more fundamental than language, so most likely we need that in place before we can get AI to be able to effectively understand language.
I agree. It seems to me that before language can develop you must have ideas, such as notions of space and time and existence, and in order to have ideas you probably need some kind of sensory apparatus that tells you things about yourself and about the environment you are in. But since we don't really understand how these things work, I doubt that we'll be able to recreate anything like it artificially.
We could call this the "Kantian Space-Time Assumption" as a precondition for strong AI. :) As others have mentioned, it is a philosophical question and probably one of the fundamental questions of our and next generations.
Re: AI’s Language Problem
#99Earlier quoted context omitted.
Why not simply simulate the body (or indeed multiple bodies) in a virtual world? Easier to build, parallelize, extend, maintain. Possibly somewhat safer too.
That works, if your virtual world is good. But that's a big if. You've just taken one really hard problem (learning about the world) and turned it into an even harder problem (simulating the world).
Some games are pretty convincing sand boxes and it's not obvious to me that you really need full range of senses to properly teach the AI.
Re: AI’s Language Problem
#100I don't think the issue has anything to do with cognition and more to do with something that we do so subconsciously we don't always notice it as we do it: error correction and context setting. A big part of language are our error correction channels. In text it's a lot less obvious because we twist the language to clear things up, but speech is full of a lot of "I'm sorry, what?" and "uh, you know" and hand gestures and furrowed brows and a million other side channels to get someone to repeat something or elucidate it or set a deeper context.
But that happens in text too: we group things into paragraphs and add a lot of punctuation and as we read we sometimes skim a bit, return as needed, reread what we missed the first time. (Or in texts/IMs our cultures are in the process of building whole new sub-dialects of error correction codes like emoji and "k?".)
A lot of people would think a machine is broken if it hemmed and hawed as much as people do in a normal conversation; if it needed full paragraphs of text to context set and/or explain itself.
The biggest thing lacking in voice recognition right now is not the lack in word understanding or any of the other NLP areas of research: it's in a lot of the little nuance of conversation flow. For now, most of the systems aren't very good at interruptions, for instance. From the easy like "let me respond to your question as soon as I understand what you are asking to save us both time" to the harder but perhaps more important things like "No [that's not what I mean]" and "Wait [let me add something or let me change my mind]" and "Uh [you really just don't get it]" and presumably really hard ones like clears throat [listen carefully this time].
The point should not be that we hit 100% accuracy: real people in real conversations don't have 100% accuracy. The issue is how do you correct from the failures in real time and keep that "conversational" without feeling strained or overly verbose (such as the currently common "I heard x, is that correct?" versus "x?" and head nod or very quick "yup").
We don't consciously think about the error correction systems in play in a conversation so that makes them hard to judge/replicate and it's easy to imagine there's an uncanny valley waiting there for us to get from no "natural error correction" ability across to supporting error correction in a way that it works with our natural background mechanisms.
At least in my mind, that's probably the next big area to study in language recognition is deeper looks into things like error correction sub-channels and conversational timing (esp. interruption) and elocution ("uh", "um", "you know", "that thing", "right, the blue one"). I'd even argue that what we have today is probably already getting to "good enough" for the long run if it didn't require us to feel like we have to be "so exact" because you only get one sentence at a time and you don't have good error correcting channels with what we have today.