We don't marvel over the ATM reading a handwritten paper check correctly. That's a considerable achievement.
Artificial Intelligence Software Is Booming, But Why Now?
21–30 of 141 posts
Re: Artificial Intelligence Software Is Booming, But Why Now?
#22I'm still not comfortable calling the thing that is booming "artificial intelligence". It is mostly pattern recognition and classification. Intelligence is something else.
Intelligence is the ability for and speed of processing new information, at least as it's measured by metrics like IQ.
Sometimes AI/ML is directly modeled after human/animal thinking (decision trees, neural networks, reinforcement learning) and sometimes it's not (Bayes' theorem, regression, general stats). What constitutes intelligence still boils down to being able to correctly and quickly process new incoming information based on prior information. I believe it definitely qualifies.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#23I'm still not comfortable calling the thing that is booming "artificial intelligence". It is mostly pattern recognition and classification. Intelligence is something else.
Yes, "signs of intelligence" include creativity, pedagogy, ..., and of course, getting bitten by the bug of existential angst.
Accept no subsititute.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#24Earlier quoted context omitted.
Convolutional layers have been around longer than that and don't have much to do with the vanishing gradient problem, which was never that big a problem to begin with. ReLUs are helpful, but they do not lead to THAT big of a speedup. Convolutional layers have definitely had a big impact for accuracy in image problems and sound problems. I think #2 is very important; they speed up both training and evaluation by a fac…
Consciousness of Things is an inevitable conclusion for humanity. The singularity is upon us with the advent of man merging into machine. We are driving that growth to go to the next level. Also, containerization.
Most of what we call 'AI' today is just fancy regression methods. No need to be so grandiose.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#25I'm still not comfortable calling the thing that is booming "artificial intelligence". It is mostly pattern recognition and classification. Intelligence is something else.
Intelligence is not mostly pattern recognition and classification? I can't imagine what you'd think it could be that wouldn't ultimately be called both of these things.
Output - the optimization of behavior. Though we're doing pretty well in that regard too...
Re: Artificial Intelligence Software Is Booming, But Why Now?
#26I'm still not comfortable calling the thing that is booming "artificial intelligence". It is mostly pattern recognition and classification. Intelligence is something else.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#27Because it's been just about 10 years since they figured out how to: 1) Use convolutional layers, ReLUs, and a few other tricks to ameliorate the vanishing gradient problem, 2) Perform continuous, high-dimensional stochastic gradient descent on graphics cards, and 3) Apply these things via stochastic grad-student descent to sufficiently massive datasets that even the most brute-force models and training methods (back…
I'm concerned about something similar, where a lot of AI techniques seem to be rushing toward a local maximum: * AI researchers do things that get good results faster out of Nvidia cards * Nvidia makes their cards faster at the things AI researchers are doing It's getting good results. We sure are going up this gradient quickly. But I don't think it's going to get us to a global maximum.
I agree w/ you about moving up a gradient quickly w/ the "GPU manufacturing deep learning research" feedback loop. I think it could last a while though. One really important area of research is figuring out how to take better advantage of greater capacity. Also, how to do more with fewer training samples (0 shot, 1 shot, etc learning). Then there's reducing the precision of the units you're using to increase capacity through software. Applying these algos to video, audio, media generation, and others will eat up all the resources you can throw at it; the algos today could take advantage of larger capacity when applied to time-series. There's so much going on that I don't see it slowing down for at least 3-5 years.
Also, I'd like to point out that we've seen some big breakthroughs in the past 10 years. But for the past 10 years, the whole field of deep learning has been looked at with skepticism and has been very niche. Over the past couple of years, a lot of money and resources have been put in place to pursue this area of research. More money doesn't necessarily mean more results, but there are many many more people working on these problems than ever before. A lot of them are legitimately brilliant researchers in the prime of their careers. I think there's still more to see.
I am concerned about an Nvidia monopoly around deep learning hardware. They give away tons of free cards to deep learning research groups, but at some point they'll want people to start buying. I assume they expect that will be the enterprise set, but if suddenly they manage to move all their capable deep learning cards to Teslas only (which have a huge markup), it will put the hobbyist deep learning developer at a big disadvantage. The only check and balance on that is the fact that Nvidia makes cards for gamers too, and gaming card competition is still somewhat robust, so any technology that gives enterprise a big boost will probably make its way to their flagship gaming cards quickly. Nvidia's only real competition, AMD, is so far behind that they might as well not be in the business. As someone who usually roots for the underdog, it pains me to see AMD fumble so badly in this whole area.
Quantum computing could be a new hill, but I think that's a ways out and I don't know enough about the topic to speak with any real confidence.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#281. Availability and accessibility of large amounts of training data. Without this training and validation is expensive if not impossible. Now if you don't have the data you can acquire it yourself. Leading to ...
2. Computational speed & storage upgrades. This applies largely to physical, time-critical things like automated driving. The self-driving car could have had all the data it needed in 1980 to do its thing, but required fast computers and lots of data storage to do it safely in real time in a feasible commercial product.
3. Advancement of algorithms. Fervor and excitement around AI/ML has been on a slow but perhaps exponential burn. This has led to the refinement of algorithms that largely sat dormant from the late 80s (and earlier) until fairly recently. This also means lots of open source libraries for people who wish to implement without caring about the underlying mechanisms behind the algorithms.
These things are leading more people to dabble recreationally and commercially.
Re: Artificial Intelligence Software Is Booming, But Why Now?
#29Ready access to high-quality usecases and training data, along with shared knowledge of the methods that work well on these helps: https://www.kaggle.com/competitions?sortBy=numberOfTeams&gro...
(disclaimer: I work at Kaggle)
Re: Artificial Intelligence Software Is Booming, But Why Now?
#30I'm still not comfortable calling the thing that is booming "artificial intelligence". It is mostly pattern recognition and classification. Intelligence is something else.
https://en.m.wikipedia.org/wiki/AI_effect
There's a similar saying I like (don't know where it's from, if anywhere)
"Artificial intelligence is a group of problems that we don't know how to solve. As soon as we know how to solve one, it gets a name and is no longer AI."