Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Difference Between AI, Machine Learning, and Deep Learning

blogs.nvidia.com

Post reply on HN
Print view View original

51–60 of 107 posts

Re: The Difference Between AI, Machine Learning, and Deep Learning

#51

Post by jules »

mjn wrote: post #46
jules wrote: post #42

Earlier quoted context omitted.

A chess computer isn't magic because you know that it is basically searching the game tree very fast with some heuristics. Neural networks are not like that. You know that it's a network of activation functions, but how that solves so many AI tasks is still more or less a mystery. So maybe the AI effect will be less strong with deep learning.

I disagree that neural networks are necessarily mysterious. In most cases they can be straightforwardly understood as brute-force function approximation (approximation here meaning mostly interpolation, in some cases capable of a limited degree of extrapolation). You take many samples of a function (either because you have a huge data set, or because you can actually sample with a simulator), and incrementally modify…

Of course, but compared to how chess algorithms work it remains far more mysterious how exactly that produces an english description of a grid of pixel values.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#52

Post by visarga »

jules wrote: post #42
Aissen wrote: post #27

Earlier quoted context omitted.

If it's magic, it's AI; if it's understood, it's algorithms.

A chess computer isn't magic because you know that it is basically searching the game tree very fast with some heuristics. Neural networks are not like that. You know that it's a network of activation functions, but how that solves so many AI tasks is still more or less a mystery. So maybe the AI effect will be less strong with deep learning.

> A chess computer isn't magic because you know that it is basically searching the game tree very fast with some heuristics

Yep, we know, the consecrated expression for this point is "they just used clever tricks".

But at some point these clever tricks add up to something akin to imagination or intuition. Do you think humans are not made of "clever tricks" too? Could it be possible that we humans have the magic fairy dust of real intelligence sprinkled in our brains and machines are lesser than us?

Re: The Difference Between AI, Machine Learning, and Deep Learning

#53

Post by visarga »

reacweb wrote: post #20
argonaut wrote: post #13

Earlier quoted context omitted.

I hear this repeated a lot. But I've never actually seen a source from, say, the 90's or the 2000s, actually saying something like "if we're able to classify images better than humans, that's true AI." I suspect it's just hearsay that's been repeated over and over again.

In the 90's, John McCarthy was calling Go "the new Drosophila of AI". Now, we have AlphaGo that is "just" a bunch of "generic" deep learning with MC. Maybe John McCarthy was right and the ideas behind AlphaGo are the first steps toward true AI. I am very impressed.

The guys at NVIDIA did something very cool recently. They took a simple CNN (convolutional neural net) and in 2-3 days they taught it to drive a car.

Article: How Our Deep Learning Tech Taught a Car to Drive - https://blogs.nvidia.com/blog/2016/05/06/self-driving-cars-3...

These neural nets are really smart, and we don't know exactly how they work, we know only in principle. But if we asked the guy who made the self driving car what is the role of the 17th neuron in the 14th layer, he would probably have no idea. Just like human brains evolve through learning, so do neural nets. Yet many people think they are just clever tricks and not truly intelligent.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#54

Post by visarga »

thallukrish wrote: post #21

If I need to show 1 million cat images to train a neural net to see cats, I wonder how a human brain can figure out cats of any kind just by seeing one or two. Is there something fundamental we are missing in going about building these deep learning stuff ?

> I wonder how a human brain can figure out cats of any kind just by seeing one or two.

A human child receives images on the retina at 20fps, say ... for 12 hours a day, for many years. That is a lot of training, a lot of images received by the brain.

What the kid does when we show it something new is to do a kind of fine-tuning of its neural net where previous visual experience is reused in order to quickly learn new types of objects. It's called one shot learning and it can be done in neural nets too.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#55

Post by yvdriess »

unabst wrote: post #45

There's that one iconic image of neurons suspended in space with bolts of electricity flashing between them. We are told that's how our brains work. We are then shown a diagram by a computer scientist. Instead of cells and thunder, we see circles and arrows. Then we are told there is an algorithm that simulates what the brain does. Viola, we have our artificial neural network. Not only do they look similar, they have…

Actual neural network papers are far removed from the impressions you get from popular science articles. Biological neurons were some inspiration for the abstraction, but the underlying model is more a direct evolution of the maths/stastical models. Griping that DNNs are not actually like neurons is akin to saying that Object Oriented programming is nothing like the organisation of biological cells it was inspired on. A useful abstraction is a useful abstraction, there is little reason to think that making the computational models any closer to physical processes is going to bring any improvements.

Neuromorphic computing is the field that tries to more accurately mimic spiking neurons, but making something useful out of it takes a backseat. It is still an open question if it is going to be useful.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#56

Post by c0g »

visarga wrote: post #53
reacweb wrote: post #20

Earlier quoted context omitted.

In the 90's, John McCarthy was calling Go "the new Drosophila of AI". Now, we have AlphaGo that is "just" a bunch of "generic" deep learning with MC. Maybe John McCarthy was right and the ideas behind AlphaGo are the first steps toward true AI. I am very impressed.

The guys at NVIDIA did something very cool recently. They took a simple CNN (convolutional neural net) and in 2-3 days they taught it to drive a car. Article: How Our Deep Learning Tech Taught a Car to Drive - https://blogs.nvidia.com/blog/2016/05/06/self-driving-cars-3... These neural nets are really smart, and we don't know exactly how they work, we know only in principle. But if we asked the guy who made the self…

That's because they're not intelligent. They're layered function approximators with some specialisations for vision - conv layers. Their structure is designed by humans as is the data they are fed with. Maybe the human brain is the same, IMO it isn't. Humans seek out new experiences and can introspect and at their best assess their level of knowledge of a sibject. DNNs cannot.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#57

Post by booli »

bpizzi wrote: post #37
vidarh wrote: post #26

Earlier quoted context omitted.

> Even a human brain has to train for ~4-5 months to become interested in shapes Everyone who wonder how (on a superficial level) grown up humans are so good at learning new categories really should spend time around babies and toddlers and children for this reason... You quickly realise how much training and brain development it actually takes before we're capable of doing much.

By far the greatest part of the time I spent with my two toddlers (5 and 3) is spent repeating the same things, again and again, and trying to find shorcuts for learning (which shorcuts might just not exist, afaik). Then one day they start to get it (like "fire burns"), but it's still not there for sure until they experiment it deeply multiple times. The dev in me can't help but see this two little humans as big migh…

And this tuning is of utmost important. People often forget that the tuning is human intervention with specific 'specs'. The little baby machine get's quickly tuned to these, good or bad.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#58

Post by visarga »

denzil_correa wrote: post #34
oliwaw wrote: post #4

Deep learning is just a rebranding of "neural networks". When neural nets became unpopular in the 90s and early 2000s, people talked about "multilayer networks" (dropping the "neural") since it wasn't really useful to think about this approach from the neuro perspective (since it's such a cartoonish model of real neural networks anyway). Now that very deep networks have become possible, and various graphical models a…

Basically, Deep Learning is now back on the map due to two things - (1) Large amounts of data available and (2) Faster processing power viz. GPUs.

Well, yes, mostly, but there also have been genuine discoveries in the last 10 years. We can now train deep networks because we learned how to regularize - before it was impossible because of vanishing gradients. We can have even 1000-layer deep nets, which would have been unthinkable. Also, there are some interesting approaches to unsupervised learning like GANs and VAEs. We learned how to embed words and not only words, but multiple classes of things into vectors. Another one would be the progress in reinforcement learning, with the stunning success of AlphaGo and playing over 50 Atari games. Current crop of neural nets do Bayesian statistics, not just classification. We are playing with attention and memory mechanisms in order to achieve much more powerful results.

Re: The Difference Between AI, Machine Learning, and Deep Learning

#60

Post by joakleaf »

maaaats wrote: post #19
oliwaw wrote: post #4

Deep learning is just a rebranding of "neural networks". When neural nets became unpopular in the 90s and early 2000s, people talked about "multilayer networks" (dropping the "neural") since it wasn't really useful to think about this approach from the neuro perspective (since it's such a cartoonish model of real neural networks anyway). Now that very deep networks have become possible, and various graphical models a…

Yeah, I've always heard that expert systems are good, because there you can reason about the solution. For instance diagnosing people based on rules contributed by doctors. You can trace the steps the algorithm takes easily. But for a neural net, you cannot say why this particular net should be trusted, as you don't know how it arrives at a solution. Therefore it's "scary" to use. While I don't agree, it explains why…

Random Decision Forests provide sort of a middle ground.

For training, a large set of decision trees are built randomly based on the input features.

When classifying input for one tree, each node considers feature value of the input, and decides on a branch. Leafs corresponds to a classification, so when a leaf is reached, the tree has classified the given input.

By having a large set of trees, and picking e.g. the most common resulting class (majority vote), we increase accuracy.

However, each individual tree can actually be reasoned about. E.g. you can see the analysis (nodes) leading to each class (leafs).

I've had some success with RDFs in the past, and highly recommend them!

They are very easy to implement, very efficient to train and query, and they seem to work really great on classification of "discrete input" (i.e. where the input feature values are binary or from relatively small sets).

Post reply on HN