Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI’s Language Problem

technologyreview.com

Post reply on HN
Print view View original

81–90 of 244 posts

Re: AI’s Language Problem

#81

Post by djsumdog »

tormeh wrote: post #71
dmreedy wrote: post #33

Earlier quoted context omitted.

Yep. The problem is that it's _so_ noisy, that the encryption, as it were, might be too strong to crack with statistical methods. You might need the key; i.e., something like a human brain. EDIT: a combination of noise, I should say, and paucity of information.

No, you only need context. Context in the form of knowledge about the place, company and history that the statement is spoken in. Wikipedia will serve well for a lot of that.

But that's the thing with AI. We make the context. In the case of AlphaGo, IBM's Watson, Self driving cars, we set the goal. There are different heuristics, but we always need to define what is "right" or what the "goal" is.

For AI to determine their own goals, well now you get into awareness ... consciousness. At a fundamental mathematical level, we still have no idea how these work.

We can see electrical signals in the brain using tools and know it's a combination of chemicals and pulses that somehow make us do what we do ... but we are still a long way from understanding how that process really works.

Re: AI’s Language Problem

#82

Post by hyperion2010 »

I have often wondered whether the reason why 'AI' has struggled with human language is because most programs are not embodied. If you cannot jump when you hear the word joy, and cannot cry when you hear the word death and feel the other sensations that go with them, you cannot truly understand the word. The lack of additional visual cues and context will continue to severely hamper the ability for machines to correctly understand the semantics of human language. I know the language guys like to work with their strings of characters, but humans can only communicate with those because we have already built our semantic framework by being active agents in the world.

Re: AI’s Language Problem

#83

Post by Florin_Andrei »

brianchu wrote: post #8

Deep learning has succeeded tremendously with perception in domains that tolerate lots of noise (audio/visual). Will those successes continue with perception in domains that are not noisy (language) and inference/control , which the article touches on? I think it really is unclear whether those challenges will require fundamental developments or just more years of incremental improvement. If fundamental developments…

> Deep learning has succeeded tremendously with perception in domains that tolerate lots of noise (audio/visual). Will those successes continue with perception in domains that are not noisy (language)

I wonder if it's just a different kind of "noise". Higher level, more structured.

> My guess is we may need a fundamental breakthrough in a newfangled hierarchical learning system that is better suited for language to “solve” NLP.

It seems fairly evident that there are many hierarchies inside the brain, each level working with outputs from lower-level processing units. In a sense, something like AlphaGo is hierarchy-poor - it has a few networks loosely correlated with a decision mechanism.

But the brain probably implements a "networks upon networks" model, that may also include hierarchical loops and other types of feedback.

I think, to have truly human level NLP, we'd have to simulate reasonably close the whole hierarchy of meaning, which in turn is given by the whole hierarchy of neural aggregates.

Re: AI’s Language Problem

#84

Post by erikpukinskis »

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they will only be able to understand us to the extent that they have body parts and social experiences that are analogous to ours.

This isn't an engineering problem, it's a philosophical problem: We are blind to most things. We can only see what we can relate to personally. We can use language and other symbol systems to expand our understanding by permuting and recombining our personal experiences, but everything is grounded in our interactive developmental trajectory.

The kitten-in-a-cart experiment demonstrates this clearly: http://io9.gizmodo.com/the-seriously-creepy-two-kitten-exper... Interaction is crucial for perception. Sensation is not experience.

And here's the rub: Once you give an AI an animal-like personal developmental trajectory to use for grounding the semantics of their symbol systems, you end up with something which is not particularly different or better than a human cyborg.

Re: AI’s Language Problem

#85

Post by Florin_Andrei »

hacker42 wrote: post #78
brianchu wrote: post #8

Deep learning has succeeded tremendously with perception in domains that tolerate lots of noise (audio/visual). Will those successes continue with perception in domains that are not noisy (language) and inference/control , which the article touches on? I think it really is unclear whether those challenges will require fundamental developments or just more years of incremental improvement. If fundamental developments…

There is evidence that language is fairly smooth though. For example, we can extract e.g. the gender vector from a word embedding space that is learned by a recurrent neural network. That seems to hint at the possibility that words, sentences and concepts live in smooth, high-dimensional manifold that makes them learnable for us in the first place (because in that case they can be learned by small local improvements…

> I have a pet theory that the discovery of sharp features and boolean programs might heavily rely on noise. If the error surface becomes too discrete, we basically need to backup to pure random optimization (i.e. trying any direction by random chance and keep it, if it is better). That allows us to skip down the energy surface even without the presence of a gradient.

Isn't that basically Monte Carlo?

Re: AI’s Language Problem

#86

Post by heimatau »

artursapek wrote: post #75

I wonder if in the future we'll have matrix-style "blobs" of knowledge that we can plug into compatible AI systems. That way it has to just be trained once on a strong system, and then other systems can take advantage of the learning by forking the state and importing it. It would definitely speed up training AI, and possibly even enable it on lower powered hardware. Imagine downloading "english teenager slang 2016 v…

I think this is too linear. My two cents are that AI, when it cracks one language, many languages will be cracked after that. In other words, exponential growth. With Open Source still being prevalent, I just think the libraries will be shared as it will bring in a new era for humanity. Think about all those languages we don't even have translators for (namely those countries that have only 20 people speaking a specific derivation of a derivation of a .....of a language humanity can translate). There are almost 7,000 languages in the world [1]. Language is a tough challenge but when we find a breakthrough, it's my strong opinion, that we'll bring everyone to the table of communication.

[1] http://www.linguisticsociety.org/content/how-many-languages-...

Re: AI’s Language Problem

#87

Post by outsideline »

wildmusings wrote: post #25

Consciousness is the secret sauce. Consciousness as in "subjective experience", what separates us from philosophical zombies, the sensation of the color blue, of middle-c. Our bodies evolved and kept this extremely rich phenomenon for a reason; it is extraordinarily unlikely for it to have arisen and remained through genetic drift alone. My theory, and I'd love to find someone offering a similar and more fleshed-out…

It won't remain a taboo when someone proofs a functional computational model for it. It seems that will need to occur to silence the idea that its quackery. Nothing short of that will suffice. Seeing will indeed be believing.

I was partly inspired to pursue the path I took in R&D by observing that the industry didn't seem to want to consider someone thinking different or working on the true foundation of A.I (the hard problems).

I figured, if I was able to write software up and down the stack for the billion dollar network infrastructure equipment that powers and services the internet, I probably knew what I was doing w.r.t to engineering.

A networked system with a missing foundation...The rest is history and I look forward to making disclosures about my work in the near future.

In the meantime, you should know that there are quite capable and industry proven individuals working on this. They aren't quacks, maintain graduate degrees from the top universities of America, and have a proven track record in the industry. The spotlight just doesn't shine in their direction. Of course, once a functional model is proof'd, I'm sure that will change. Such is the history of new paradigms and those who, through deep and new understanding, seek to usher them in...

Re: AI’s Language Problem

#88

Post by hacker42 »

Florin_Andrei wrote: post #85
hacker42 wrote: post #78

Earlier quoted context omitted.

There is evidence that language is fairly smooth though. For example, we can extract e.g. the gender vector from a word embedding space that is learned by a recurrent neural network. That seems to hint at the possibility that words, sentences and concepts live in smooth, high-dimensional manifold that makes them learnable for us in the first place (because in that case they can be learned by small local improvements…

> I have a pet theory that the discovery of sharp features and boolean programs might heavily rely on noise. If the error surface becomes too discrete, we basically need to backup to pure random optimization (i.e. trying any direction by random chance and keep it, if it is better). That allows us to skip down the energy surface even without the presence of a gradient. Isn't that basically Monte Carlo?

It's called random optimization or random search depending on whether you sample from a normal or uniform distribution for the random direction. MC typically refers to any algorithm that computes approximated solutions using random numbers (as opposed to Las Vegas algorithms which use random numbers to always compute the correct solution). So, yes, RO, RS and gradient descent are MC local optimization algorithms.

Re: AI’s Language Problem

#89

Post by byebyetech »

dmreedy wrote: post #19
grondilu wrote: post #5

It seems to me that a full mastery of language requires a grasp of semantics, that is the ability to understand what a sentence means. I doubt it's possible to do that without having basic common sense along with an overall representation of the world, and that looks very close to strong AI, imho. So I'm not surprised computers keep on struggling with language applications. Once they succeed strong AI will not be muc…

I think the 'overall representation of the world' requirement is pretty key here. Language in AI is often treated as its own class of problem, with the assumption that there is somehow enough signal in the raw mess of examples provided to any given learning system (usually just plain text, stripped of any prosody, emotion, cultural context, imagery; any of the other modalities of communication available to a demonstr…

>>I think that language is what the human brain does.

I think language is a UI with our own brain. It allows us to interact with its knowledge system and representation of the world. Self is a thin client running on the vast knowledge system. If you think about it, thinking is not where the real thinking happens. We get intuition signals from the brain on what is true / false , which are required for our higher level thinking. So thinking we do is also a thin client running on top of the Brain OS. Both the thinking and language are serialization tools of representation of the world that was solely evolved for communication with other brains. Since we don't have direct neural link with other brains, we have to serialize it and hence language based thinking.

So i think to evolve language understanding in machines, we might have to simulate many intelligent agents in a simulated environment and let them collaborate. Similar to how our brains collaborated and gave rise to natural languages.

Re: AI’s Language Problem

#90

Post by protomikron »

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

> This isn't an engineering problem, it's a philosophical problem [...]

Indeed. However we can not rule out the possibility that we engineer a system without a "body" (I think you are referring to the Embodied mind thesis?). It is a complicated topic and discussions about it are futile without precise definitions of loaded terms, like "intelligence", "body", etc.

A rather well-defined test is the classic Turing test and I wouldn't dismiss the possibility that it can be passed by a bodyless machine/program-thing.

Post reply on HN