The Difference Between AI, Machine Learning, and Deep Learning • Hacker News Board

Re: The Difference Between AI, Machine Learning, and Deep Learning

#61

Post by cgearhart » Fri, Aug 19, 2016, 1:47 PM UTC

ktta wrote: post #12

Also, an interesting read related to this would 'the AI effect'[0]. A lot of the stuff Deep Learning/Machine Learning is able to do today would be looked at as something that only 'true' AI (whatever consensus on what that means is; I think of it as AGI) would be able to do. But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was sol…

I would argue that there is actually an unrecognized inverse of this effect, too. People have a tendency to conflate domain-specific human-level performance with "intelligence" (for some definition of that word). Then suddenly we jump to "DeepMind won at go, therefore humanity is on the brink of creating Skynet." Deep learning has been like this so far, because it works in many new domains that have resisted previous techniques like image recognition, speech recognition, go, etc.

The "AI Effect" -- saying, "it's just an algorithm" once we succeed -- is an artifact of developing AI by using it as an algorithm for a domain-specific task. It is "just" an algorithm, because that's all it needs to be to win at go, identify cats, or whatever. Fundamentally, it's not very surprising that when you set out to make a system that's really good at playing go...you end up with a system that's really good at playing go. Of course it's hard -- that's what we should appreciate -- but most of the problems we consider "hard" are based on what we think it is hard for a computer to do in order to solve the problem. "Searching the space of all possible go moves is computationally intractable, therefore this problem is hard." In reality, the problem may not be as hard as we think if the agent doesn't have to search the space of all possible moves.

"Hard" problems are multi-optimization, where even the definitions aren't very clear: "learn as much as possible and live a happy life while making a productive living for yourself". Turn that into an objective function...

Re: The Difference Between AI, Machine Learning, and Deep Learning

#62

Post by TheOtherHobbes » Fri, Aug 19, 2016, 1:54 PM UTC

ansible wrote: post #47

thallukrish wrote: post #21

If I need to show 1 million cat images to train a neural net to see cats, I wonder how a human brain can figure out cats of any kind just by seeing one or two. Is there something fundamental we are missing in going about building these deep learning stuff ?

This difference is due to deep learning not being able to build models. Check out the recent paper "Building Machines That Learn and Think Like People": https://arxiv.org/abs/1604.00289 Humans can see one or a few examples of a novel object, such as a cat, and create a fully 3D mental model of it. So we know what it will look like in different orientations and lighting conditions.

I don't think we have a fully 3D mental model. I think we have a set of core invariants that are constant across a wide variety of conditions.

I.e. we can recognise static 2D photos of cats, 2D movies of cats, 3D movies of cats, and real cats in the real world.

The invariants are probably relatively simple - a set of head geometries and head feature shapes/distribution, with some secondary colour and texture confirmation.

What's interesting is that we can recognise modifiers to the invariants - e.g. a shaved cat is still recognisably a cat, but parsed as "cat without fur."

We can also recognise invariants when they're pared down to essentials in cartoons and sketches.

https://www.youtube.com/watch?v=R9qdyXCVNVk

A lot of learning is really just data compression - finding a minimal set of low-resource invariant specifics from a wide range of noisy high-resource inputs.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#63

Post by qntty » Fri, Aug 19, 2016, 2:09 PM UTC

bogomipz wrote: post #44

I was confused by this sentence in the last paragraph: "Deep Learning has enabled many practical applications of machine learning and by extension the overall field of AI." Is it not the reverse - machine learning has enabled deep learning? Can someone comment on how the two - machine learning and Deep learning relate? Is the relationship sequential i.e a data set from machine learning is the the input for a neural n…

the relationship suggested by the article is that deep learning is a subset of machine learning

Re: The Difference Between AI, Machine Learning, and Deep Learning

#64

Post by bogomipz » Fri, Aug 19, 2016, 2:26 PM UTC

qntty wrote: post #63

bogomipz wrote: post #44

I was confused by this sentence in the last paragraph: "Deep Learning has enabled many practical applications of machine learning and by extension the overall field of AI." Is it not the reverse - machine learning has enabled deep learning? Can someone comment on how the two - machine learning and Deep learning relate? Is the relationship sequential i.e a data set from machine learning is the the input for a neural n…

the relationship suggested by the article is that deep learning is a subset of machine learning

Sure, that was kind of my point, and as such I would think the sentence should instead read

"Machine Learning has enabled many practical applications of deep learning .."

No?

Re: The Difference Between AI, Machine Learning, and Deep Learning

#65

Post by cscurmudgeon » Fri, Aug 19, 2016, 2:35 PM UTC

seanwilson wrote: post #18

ktta wrote: post #12

Also, an interesting read related to this would 'the AI effect'[0]. A lot of the stuff Deep Learning/Machine Learning is able to do today would be looked at as something that only 'true' AI (whatever consensus on what that means is; I think of it as AGI) would be able to do. But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was sol…

> But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states. I think the issue is that current examples of AI can mostly only solve problems in a single domain instead of in a variety of domains. Deve…

> I think once we have a comprehensive AI program that can play many kinds of games, can have conversations, can complete school exam questions, can write stories etc. the question of what intelligence is will become more interesting.

Or even just talk about Go, answer complex questions about it in natural language, and state and prove interesting mathematical results about the complexity of Go.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#66

Post by [deleted] » Fri, Aug 19, 2016, 2:38 PM UTC

visarga wrote: post #48

seanwilson wrote: post #18

Earlier quoted context omitted.

> But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states. I think the issue is that current examples of AI can mostly only solve problems in a single domain instead of in a variety of domains. Deve…

> once we have a comprehensive AI program that can play many kinds of games Before AlphaGo, DeepMind released a reinforcement learning algorithm that could play many Atari games just from the raw pixels on the screen, in many games surpassing humans. The same algorithm. https://arxiv.org/pdf/1312.5602.pdf Reinforcement learning is a general framework for learning behavior from acting in an environment with the purpos…

[deleted]

Re: The Difference Between AI, Machine Learning, and Deep Learning

#67

Post by cscurmudgeon » Fri, Aug 19, 2016, 2:38 PM UTC

visarga wrote: post #48

seanwilson wrote: post #18

Earlier quoted context omitted.

> But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states. I think the issue is that current examples of AI can mostly only solve problems in a single domain instead of in a variety of domains. Deve…

> once we have a comprehensive AI program that can play many kinds of games Before AlphaGo, DeepMind released a reinforcement learning algorithm that could play many Atari games just from the raw pixels on the screen, in many games surpassing humans. The same algorithm. https://arxiv.org/pdf/1312.5602.pdf Reinforcement learning is a general framework for learning behavior from acting in an environment with the purpos…

> but that they used a general method that can be used to do other tasks as well. It is not a limitation of AI that we generally make systems that are good at only one thing

This is exactly the kind of hubris and reasoning that brought down good old fashioned AI initially. They also had a lot of initial impressive wins with methods that looked quite general at that time.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#68

Post by visarga » Fri, Aug 19, 2016, 3:03 PM UTC

cscurmudgeon wrote: post #67

visarga wrote: post #48

Earlier quoted context omitted.

> once we have a comprehensive AI program that can play many kinds of games Before AlphaGo, DeepMind released a reinforcement learning algorithm that could play many Atari games just from the raw pixels on the screen, in many games surpassing humans. The same algorithm. https://arxiv.org/pdf/1312.5602.pdf Reinforcement learning is a general framework for learning behavior from acting in an environment with the purpos…

> but that they used a general method that can be used to do other tasks as well. It is not a limitation of AI that we generally make systems that are good at only one thing This is exactly the kind of hubris and reasoning that brought down good old fashioned AI initially. They also had a lot of initial impressive wins with methods that looked quite general at that time.

The difference being that this time there are real applications to deep learning.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#69

Post by eli_gottlieb » Fri, Aug 19, 2016, 3:11 PM UTC

ktta wrote: post #12

Also, an interesting read related to this would 'the AI effect'[0]. A lot of the stuff Deep Learning/Machine Learning is able to do today would be looked at as something that only 'true' AI (whatever consensus on what that means is; I think of it as AGI) would be able to do. But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was sol…

>But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states.

Well, the problem here is that we keep expecting "real intelligence" to have magical properties: we expect intelligence to be a way of creating new and interesting thoughts, ex nihilo, rather than an efficient engine for distilling structure from experiences, and to be ontologically special to prove how special we humans are on a universal scale.

There are cognitive science and neuroscience labs hard at work on deciphering how actually-existing human thought works, but as far as I can tell, everyone facinated by "AI" is so enamoured with intelligence being special that they don't pay attention to those labs. A great deal of books and papers are thus published, and as far as "AI" is concerned, it's all in vain somehow.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#70

Post by throwaway729 » Fri, Aug 19, 2016, 3:16 PM UTC

jules wrote: post #51

mjn wrote: post #46

Earlier quoted context omitted.

I disagree that neural networks are necessarily mysterious. In most cases they can be straightforwardly understood as brute-force function approximation (approximation here meaning mostly interpolation, in some cases capable of a limited degree of extrapolation). You take many samples of a function (either because you have a huge data set, or because you can actually sample with a simulator), and incrementally modify…

Of course, but compared to how chess algorithms work it remains far more mysterious how exactly that produces an english description of a grid of pixel values.

Probably only because every CS undergrad or self-trained person sees at least a few search algorithms. So although they probably couldn't write a performant chess-playing program, they get the general idea.

I expect in 20 years undergraduate curricula evolve to cover NNs, and then those courses will have their salient bits adopted into mass market expository books/blog posts/code examples/etc. At that point someone like you will be saying something like you said about whatever the new hotness is.