51–60 of 141 posts
Earlier quoted context omitted.
https://en.m.wikipedia.org/wiki/AI_effect
There's a similar saying I like (don't know where it's from, if anywhere) "Artificial intelligence is a group of problems that we don't know how to solve. As soon as we know how to solve one, it gets a name and is no longer AI."
I suppose this could be the same thing.
"Democratizing AI" is the word being thrown around a lot by CEOs, PR, et al. But only time will tell whether it's true democracy.
We don't marvel over the ATM reading a handwritten paper check correctly. That's a considerable achievement.
From the first time it started doing that, I've been amazed that that works so well in a production system. Every. Time.
The US Postal Service used to have 55 centers where humans tried to read envelopes that the machines couldn't. They're now down to one. "We get the worst of the worst. It used to be that we’d get letters that were somewhat legible but the machines weren’t good enough to read them. Now we get letters and packages with the most awful handwriting you can imagine."
[1] http://www.nytimes.com/2013/05/04/us/where-mail-with-illegib...
Because it's been just about 10 years since they figured out how to: 1) Use convolutional layers, ReLUs, and a few other tricks to ameliorate the vanishing gradient problem, 2) Perform continuous, high-dimensional stochastic gradient descent on graphics cards, and 3) Apply these things via stochastic grad-student descent to sufficiently massive datasets that even the most brute-force models and training methods (back…
I've been thinking a lot about this lately, especially as I've been learning more about the difference between "nice" optimization problems with differentiable cost functions, and "messy" optimizations with less ideal cost functions. Is there any reading material you can recommend on the recent history of ML/AI for someone like me, with a dilettante's understanding of these problems and an interest in optimization?
(Disclaimer: I don't know that much about ML/Deep Learning, the above was recommended by deeplearning.net and is what I'm slogging through ATM. I do have some background in "classical" statistics & optimization theory, and the above article is not that math-heavy)
Earlier quoted context omitted.
And then there's the metric of how pitifully little intelligence we've managed to get from all those GFLOPS. I'd say that all the GFLOPS together don't add up to the intelligence of a single Portia africana. http://news.nationalgeographic.com/2016/01/160121-jumping-sp...
I don't buy that. Spiders can't beat a human at Go. Sure, they weren't evolved to do that. But how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? Whereas if you made a spider hunting video game, I bet researchers could train AI's that could beat it within a few months. Video game playing has actually become a big area of research recently an…
Earlier quoted context omitted.
I'm concerned about something similar, where a lot of AI techniques seem to be rushing toward a local maximum: * AI researchers do things that get good results faster out of Nvidia cards * Nvidia makes their cards faster at the things AI researchers are doing It's getting good results. We sure are going up this gradient quickly. But I don't think it's going to get us to a global maximum.
I agree w/ you about moving up a gradient quickly w/ the "GPU manufacturing deep learning research" feedback loop. I think it could last a while though. One really important area of research is figuring out how to take better advantage of greater capacity. Also, how to do more with fewer training samples (0 shot, 1 shot, etc learning). Then there's reducing the precision of the units you're using to increase capacity…
1. GPU as a service: While all major providers (AWS, Azure) offer Teslas on their servers, there is no AMD on the cloud (that I know of)
2. Key libraries: Nvidia comes with matrix libraries and cuDNN out of the box, while for AMD, there are only open source offerings that are a bit difficult to manage.
But, If you write your own software or rely on open source, AMD is quite performant and affordable. The problem is that it is really obscure. So, yes, they are really bad at marketing and if you are looking at them as a user instead of as a developer, they are invisible.
Earlier quoted context omitted.
And then there's the metric of how pitifully little intelligence we've managed to get from all those GFLOPS. I'd say that all the GFLOPS together don't add up to the intelligence of a single Portia africana. http://news.nationalgeographic.com/2016/01/160121-jumping-sp...
I don't buy that. Spiders can't beat a human at Go. Sure, they weren't evolved to do that. But how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? Whereas if you made a spider hunting video game, I bet researchers could train AI's that could beat it within a few months. Video game playing has actually become a big area of research recently an…
Date Approximate cost per GFLOPS inflation adjusted to 2013 US dollars --------------------------------------------------------------------------------- 1961 $8.3 trillion 1984 $42,780,000 1997 $42,000 April 2000 $1,300 May 2000 $836 August 2003 $100 August 2007 $52 March 2011 $1.80 August 2012 $0.73 June 2013 $0.22 November 2013 $0.16 December 2013 $0.12 January 2015 $0.06
Date Approximate cost per GFLOPS inflation adjusted to 2013 US dollars --------------------------------------------------------------------------------- 1961 $8.3 trillion 1984 $42,780,000 1997 $42,000 April 2000 $1,300 May 2000 $836 August 2003 $100 August 2007 $52 March 2011 $1.80 August 2012 $0.73 June 2013 $0.22 November 2013 $0.16 December 2013 $0.12 January 2015 $0.06
Earlier quoted context omitted.
I'm concerned about something similar, where a lot of AI techniques seem to be rushing toward a local maximum: * AI researchers do things that get good results faster out of Nvidia cards * Nvidia makes their cards faster at the things AI researchers are doing It's getting good results. We sure are going up this gradient quickly. But I don't think it's going to get us to a global maximum.
Human consciousness itself is probably a local maximum. Not necessarily a bad thing to see how far a certain process can get you as long as you keep getting results. At least until there's some other insight along the way.
