I'm still not comfortable calling the thing that is booming "artificial intelligence". It is mostly pattern recognition and classification. Intelligence is something else.
Artificial Intelligence Software Is Booming, But Why Now?
11–20 of 141 posts
Re: Artificial Intelligence Software Is Booming, But Why Now?
#12Because it's been just about 10 years since they figured out how to: 1) Use convolutional layers, ReLUs, and a few other tricks to ameliorate the vanishing gradient problem, 2) Perform continuous, high-dimensional stochastic gradient descent on graphics cards, and 3) Apply these things via stochastic grad-student descent to sufficiently massive datasets that even the most brute-force models and training methods (back…
Convolutional layers have been around longer than that and don't have much to do with the vanishing gradient problem, which was never that big a problem to begin with. ReLUs are helpful, but they do not lead to THAT big of a speedup. Convolutional layers have definitely had a big impact for accuracy in image problems and sound problems. I think #2 is very important; they speed up both training and evaluation by a fac…
Consciousness of Things is an inevitable conclusion for humanity. The singularity is upon us with the advent of man merging into machine. We are driving that growth to go to the next level.
Also, containerization.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#13Because it's been just about 10 years since they figured out how to: 1) Use convolutional layers, ReLUs, and a few other tricks to ameliorate the vanishing gradient problem, 2) Perform continuous, high-dimensional stochastic gradient descent on graphics cards, and 3) Apply these things via stochastic grad-student descent to sufficiently massive datasets that even the most brute-force models and training methods (back…
Convolutional layers have been around longer than that and don't have much to do with the vanishing gradient problem, which was never that big a problem to begin with. ReLUs are helpful, but they do not lead to THAT big of a speedup. Convolutional layers have definitely had a big impact for accuracy in image problems and sound problems. I think #2 is very important; they speed up both training and evaluation by a fac…
> they speed up both training and evaluation by a factor of 100
The factor is more like 5-10x for truly optimized CPU versus GPU in fp32 (e.g., nnpack on CPU versus cuDNN).
Re: Artificial Intelligence Software Is Booming, But Why Now?
#14I'm still not comfortable calling the thing that is booming "artificial intelligence". It is mostly pattern recognition and classification. Intelligence is something else.
ISTM you may have found the answer to TFA's question. In the early days of AI, they were looking for actual intelligence. Eventually everyone realized that won't be found soon. Today's success is mostly a successful refocusing on attainable heuristics, rather than solving many problems related to actual intelligence.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#15I'm still not comfortable calling the thing that is booming "artificial intelligence". It is mostly pattern recognition and classification. Intelligence is something else.
Intelligence is not mostly pattern recognition and classification? I can't imagine what you'd think it could be that wouldn't ultimately be called both of these things.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#16Earlier quoted context omitted.
Convolutional layers have been around longer than that and don't have much to do with the vanishing gradient problem, which was never that big a problem to begin with. ReLUs are helpful, but they do not lead to THAT big of a speedup. Convolutional layers have definitely had a big impact for accuracy in image problems and sound problems. I think #2 is very important; they speed up both training and evaluation by a fac…
> they speed up both training and evaluation by a factor of 100 The factor is more like 5-10x for truly optimized CPU versus GPU in fp32 (e.g., nnpack on CPU versus cuDNN).
I'm basing that number on https://github.com/jcjohnson/cnn-benchmarks. I'm sure the exact number depends on the cpu/gpu/algorithm.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#17Because it's been just about 10 years since they figured out how to: 1) Use convolutional layers, ReLUs, and a few other tricks to ameliorate the vanishing gradient problem, 2) Perform continuous, high-dimensional stochastic gradient descent on graphics cards, and 3) Apply these things via stochastic grad-student descent to sufficiently massive datasets that even the most brute-force models and training methods (back…
Quantum computing can technically create a complete paradigm shift as it increases to computing power drastically. well, in theory at least
Re: Artificial Intelligence Software Is Booming, But Why Now?
#18Because it's been just about 10 years since they figured out how to: 1) Use convolutional layers, ReLUs, and a few other tricks to ameliorate the vanishing gradient problem, 2) Perform continuous, high-dimensional stochastic gradient descent on graphics cards, and 3) Apply these things via stochastic grad-student descent to sufficiently massive datasets that even the most brute-force models and training methods (back…
I'm concerned about something similar, where a lot of AI techniques seem to be rushing toward a local maximum: * AI researchers do things that get good results faster out of Nvidia cards * Nvidia makes their cards faster at the things AI researchers are doing It's getting good results. We sure are going up this gradient quickly. But I don't think it's going to get us to a global maximum.
Human consciousness itself is probably a local maximum. Not necessarily a bad thing to see how far a certain process can get you as long as you keep getting results.
At least until there's some other insight along the way.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#19Earlier quoted context omitted.
> they speed up both training and evaluation by a factor of 100 The factor is more like 5-10x for truly optimized CPU versus GPU in fp32 (e.g., nnpack on CPU versus cuDNN).
I'm basing that number on https://github.com/jcjohnson/cnn-benchmarks . I'm sure the exact number depends on the cpu/gpu/algorithm.
im2col + sgemm on CPU as in Caffe for instance is really slow; you are heavily penalized for extra memory traffic and the sgemm tile sizes are probably not well tuned for the problem size at hand.
At the roofline of performance, the difference in both mem b/w and arithmetic throughput between CPU and GPU is only 5-10x (for fp32, Pascal fp16 is a different story of course), and proper implementations on the CPU will get you there.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#20I'm still not comfortable calling the thing that is booming "artificial intelligence". It is mostly pattern recognition and classification. Intelligence is something else.
ISTM you may have found the answer to TFA's question. In the early days of AI, they were looking for actual intelligence. Eventually everyone realized that won't be found soon. Today's success is mostly a successful refocusing on attainable heuristics, rather than solving many problems related to actual intelligence.
Love it. So we today have "A.I." and hopefully someday we may finally get A.A.I.
So what happened to Machine Learning and Computational Intelligence? Not sexy enough?