Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial Intelligence Software Is Booming, But Why Now?

nytimes.com

Post reply on HN
Print view View original

71–80 of 141 posts

Re: Artificial Intelligence Software Is Booming, But Why Now?

#71

Post by dragandj »

dharma1 wrote: post #67
dragandj wrote: post #56

Earlier quoted context omitted.

As someone who does (non-NN) ML on AMD, I might ask why you think that AMD is so far behind. In my experience, their GPU hardware is excellent, maybe even better than Nvidia, especially if you factor the price in. Where they ARE lagging heavily is: 1. GPU as a service: While all major providers (AWS, Azure) offer Teslas on their servers, there is no AMD on the cloud (that I know of) 2. Key libraries: Nvidia comes wit…

The key libraries part has kept me wondering. How much would it cost for AMD to assign a handful heavy duty engineers to this task (writing AMD optimised kernels for convolution etc)? Their management has been fast asleep for at least 2 years

That is a part of the problem: they assigned people for the task, and produced open-source libraries for matrices, FFT, maybe even something for DNNs. But, those are not polished much, and you have to hunt them down and install them yourself. And, they do a really bad job at marketing.

On the other hand, finding and installing those libraries is nothing compared to actually developing GPU computing software, so, as I said, if you want to program GPUs, even such scattered state of AMD platform is not that worse than Nvidia. Because, in Nvidia, you install CUDA and you get everything set up. And then - what do you do? You still have to learn a not-so-easy black art of optimizing kernels for the actual hardware.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#72

Post by Houshalter »

YeGoblynQueenne wrote: post #65
Houshalter wrote: post #43

Earlier quoted context omitted.

I don't buy that. Spiders can't beat a human at Go. Sure, they weren't evolved to do that. But how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? Whereas if you made a spider hunting video game, I bet researchers could train AI's that could beat it within a few months. Video game playing has actually become a big area of research recently an…

>> Spiders can't beat a human at Go. Finding the best next move from a given board position at Go is a single cognitive task, whereas surviving in the natural world requires an intelligence that can excel at a possibly infinite number of such tasks. Portia spiders are particularly amazing in that they can exhibit such complex behaviours as ambushing or stalking prey. Like all insects they do this sort of thing with a…

There is no mathematical distinction between different "tasks". Playing Go requires thousands or even millions of different heuristics and pieces of knowledge. Each of those is a "task" in and of itself. The same technology which beat Go could also, in principle, be used to play a real time video game that requires hunting and stalking things.

Insects are indeed tiny. But they are highly specialized. They can't learn to do things differently than they are programmed. GPUs are extremely general purpose. Not only can neural nets running on them learn many tasks, but they can execute many algorithms which aren't neural networks at all.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#73

Post by YeGoblynQueenne »

Houshalter wrote: post #43
intrasight wrote: post #40

Earlier quoted context omitted.

And then there's the metric of how pitifully little intelligence we've managed to get from all those GFLOPS. I'd say that all the GFLOPS together don't add up to the intelligence of a single Portia africana. http://news.nationalgeographic.com/2016/01/160121-jumping-sp...

I don't buy that. Spiders can't beat a human at Go. Sure, they weren't evolved to do that. But how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? Whereas if you made a spider hunting video game, I bet researchers could train AI's that could beat it within a few months. Video game playing has actually become a big area of research recently an…

>> how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go?

That's a different issue. Spiders, as all insects, don't learn.

It seems that insects have some kind of firmware-style programming that can't change, because they don't have the cognitive ability to change it. This firmware can be unbelievably complex, like in the case of Portia spiders stalking prey, but it can't develop any further, at least not on the individual level.

Even this however is nothing short of amazing. The truth is that we haven't developed a single program that can do what a Portia spider can do, and in fact learning algorithms (or strategies) like reinforcement learning are our best bet to avoid having to explicitly write down the rules for this sort of behaviour. And we want to avoid having to do that because it's probably beyond our abilities.

The learning algorithms we have right now however aren't a very good way to develop intelligence that can display such complex behaviours. Those algorithms are one-trick ponies, they learn to do one thing very well but if you want to do more than one thing you need to train more instances of the algorithm- and then you need to find a way to combine the trained instances in a cognitive architecture. And on that we haven't made much progress yet.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#74

Post by Teodolfo »

NhanH wrote: post #42
Houshalter wrote: post #37

Date Approximate cost per GFLOPS inflation adjusted to 2013 US dollars --------------------------------------------------------------------------------- 1961 $8.3 trillion 1984 $42,780,000 1997 $42,000 April 2000 $1,300 May 2000 $836 August 2003 $100 August 2007 $52 March 2011 $1.80 August 2012 $0.73 June 2013 $0.22 November 2013 $0.16 December 2013 $0.12 January 2015 $0.06

I believe the NN/ Deep learning renaissance started around 2011 -- iirc Alex net was released in 2012 and the famous Google's paper on detecting cat on youtube also around that time. It seems like the drop of costs from 2007 to 2011 is higher than the previous period. What happened in 2007? Or was it just a slow down during the start of 2000s ?

More like 2006, although there were convnets and other neural nets working well long before then of course. It just took several years for the techniques from the papers in the scientific rivial to make their way into products. The vision community was quite resistant to it, even more than the speech community.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#75

Post by dasboth »

tnecniv wrote: post #30
saosebastiao wrote: post #26

Earlier quoted context omitted.

https://en.m.wikipedia.org/wiki/AI_effect

There's a similar saying I like (don't know where it's from, if anywhere) "Artificial intelligence is a group of problems that we don't know how to solve. As soon as we know how to solve one, it gets a name and is no longer AI."

> it gets a name and is no longer AI

And that name is typically "weak AI". I wonder if we'll ever have AI that we marvel at even after we figure out how it works.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#76

Post by Houshalter »

YeGoblynQueenne wrote: post #73
Houshalter wrote: post #43

Earlier quoted context omitted.

I don't buy that. Spiders can't beat a human at Go. Sure, they weren't evolved to do that. But how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? Whereas if you made a spider hunting video game, I bet researchers could train AI's that could beat it within a few months. Video game playing has actually become a big area of research recently an…

>> how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? That's a different issue. Spiders, as all insects, don't learn . It seems that insects have some kind of firmware-style programming that can't change, because they don't have the cognitive ability to change it. This firmware can be unbelievably complex, like in the case of Portia spiders…

That's exactly my point. Spiders can't learn. Maybe they are amazing for what they do, but the comparison is apples to oranges. They do fundamentally different things.

Of course spiders are complex. Everything in biology is complex. It's been slowly refined over millions and millions of years. Countless trillions of tweaks to its DNA tested and rejected until it hit upon something that worked. No human could design something as complex as a spider.

Our artificial neural networks are just as complex though. AlphaGo probably knows millions of facts and heuristics about Go. You would never be able to understand everything it knows. Even though we didn't create the complexity ourselves, we "evolve" it just like the spider, sometimes even with algorithms based on evolution. But we have much more efficient algorithms than that, like backprop.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#77

Post by YeGoblynQueenne »

Houshalter wrote: post #72
YeGoblynQueenne wrote: post #65

Earlier quoted context omitted.

>> Spiders can't beat a human at Go. Finding the best next move from a given board position at Go is a single cognitive task, whereas surviving in the natural world requires an intelligence that can excel at a possibly infinite number of such tasks. Portia spiders are particularly amazing in that they can exhibit such complex behaviours as ambushing or stalking prey. Like all insects they do this sort of thing with a…

There is no mathematical distinction between different "tasks". Playing Go requires thousands or even millions of different heuristics and pieces of knowledge. Each of those is a "task" in and of itself. The same technology which beat Go could also, in principle, be used to play a real time video game that requires hunting and stalking things. Insects are indeed tiny. But they are highly specialized . They can't lear…

>> There is no mathematical distinction between different "tasks".

Sure there is. For instance, the minimax algorithm is a mathematical model of optimal behaviour in a specific kind of game (two-player, zero-sum, complete information games). Performing minimax to dominate such a game is a very clearly delimited task and that's exactly why a machine can do it in the first place.

More broadly, any "task" can be described mathematically by means of a program for a given machine. Saying there's no mathematical distinction between different tasks is like saying you can't write a program that does a specific thing.

>> Insects are indeed tiny. But they are highly specialized.

Agreed- see my comment above.

However, that on its own doesn't say much. Sure, insects have highly specialised firmware-style programming. How did they come to have them and how can we reproduce them? We can't hope to create such programs "by hand" so we're currently trying to create algorithms that learn them. We're still far from achieving it though.

That neural networks and other learning algorithms can learn all sorts of different things is true, however you have to train a new instance of an algorithm for every different thing you want it to learn (what I call a "task", so a new program). For each instance you need new data (and a lot of it) another huge chunk of processing power and a long time to train. Then you have to find a way to combine all the trained models into one ... thing that can operate as a coherent whole.

We haven't even started on the last bit.

Just having the technology to create pigments is far from being the same as creating a Mona Lisa.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#78

Post by singularity2001 »

YeGoblynQueenne wrote: post #73
Houshalter wrote: post #43

Earlier quoted context omitted.

I don't buy that. Spiders can't beat a human at Go. Sure, they weren't evolved to do that. But how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? Whereas if you made a spider hunting video game, I bet researchers could train AI's that could beat it within a few months. Video game playing has actually become a big area of research recently an…

>> how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? That's a different issue. Spiders, as all insects, don't learn . It seems that insects have some kind of firmware-style programming that can't change, because they don't have the cognitive ability to change it. This firmware can be unbelievably complex, like in the case of Portia spiders…

>> Spiders, as all insects, don't learn.

That is blatantly false. A recent paper has shown that bees can even be trained to distinguishing a handful of human faces( a task which they are of course not optimized for).

for starters: http://insects.about.com/od/behaviorcommunication/f/caninsec...

Re: Artificial Intelligence Software Is Booming, But Why Now?

#79

Post by Cybiote »

Houshalter wrote: post #72
YeGoblynQueenne wrote: post #65

Earlier quoted context omitted.

>> Spiders can't beat a human at Go. Finding the best next move from a given board position at Go is a single cognitive task, whereas surviving in the natural world requires an intelligence that can excel at a possibly infinite number of such tasks. Portia spiders are particularly amazing in that they can exhibit such complex behaviours as ambushing or stalking prey. Like all insects they do this sort of thing with a…

There is no mathematical distinction between different "tasks". Playing Go requires thousands or even millions of different heuristics and pieces of knowledge. Each of those is a "task" in and of itself. The same technology which beat Go could also, in principle, be used to play a real time video game that requires hunting and stalking things. Insects are indeed tiny. But they are highly specialized . They can't lear…

> The same technology which beat Go could also, in principle, be used to play a real time video game that requires hunting and stalking things.

If you generalize things enough to mean application of linear algebra, the chain rule and map, then yes that's true. However, the fact is that Neural networks still take a huge amount of skill and experimentation to design new architectures. The more powerful deeplearning becomes, the less skill will be required to do complicated things.

The AlphaGo bot was very data inefficient; at ~1 hr 18 min in http://techtalks.tv/talks/deep-reinforcement-learning/62360/, David Silver points out how they broke correlations by brute forcing millions of games while only keeping one move per game. We want something with better generalization ability than that. As amazing as AlphaGo was, it's more a very early waystation than our final destination. Recognizing the weaknesses of our models instead of glossing over them is how we get to AI!

Re: Artificial Intelligence Software Is Booming, But Why Now?

#80

Post by Cybiote »

Houshalter wrote: post #76
YeGoblynQueenne wrote: post #73

Earlier quoted context omitted.

>> how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? That's a different issue. Spiders, as all insects, don't learn . It seems that insects have some kind of firmware-style programming that can't change, because they don't have the cognitive ability to change it. This firmware can be unbelievably complex, like in the case of Portia spiders…

That's exactly my point. Spiders can't learn. Maybe they are amazing for what they do, but the comparison is apples to oranges. They do fundamentally different things. Of course spiders are complex. Everything in biology is complex. It's been slowly refined over millions and millions of years. Countless trillions of tweaks to its DNA tested and rejected until it hit upon something that worked. No human could design s…

The large majority of deployed machine learning algorithms actually can't learn. The bias is so large that most (all?) specialized hardware assumes that only inference will be done. In contrast, there is evidence for online adaptability in insects^

^ http://www.annualreviews.org/eprint/5PHddkgYYKaduPp4CcN5/ful...

Post reply on HN