Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial Intelligence Software Is Booming, But Why Now?

nytimes.com

Post reply on HN
Print view View original

91–100 of 141 posts

Re: Artificial Intelligence Software Is Booming, But Why Now?

#91

Post by Cybiote »

Houshalter wrote: post #43
intrasight wrote: post #40

Earlier quoted context omitted.

And then there's the metric of how pitifully little intelligence we've managed to get from all those GFLOPS. I'd say that all the GFLOPS together don't add up to the intelligence of a single Portia africana. http://news.nationalgeographic.com/2016/01/160121-jumping-sp...

I don't buy that. Spiders can't beat a human at Go. Sure, they weren't evolved to do that. But how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? Whereas if you made a spider hunting video game, I bet researchers could train AI's that could beat it within a few months. Video game playing has actually become a big area of research recently an…

I think you're missing the point there. Portia is able to, using just a handful of neurons (~600K), accomplish sophisticated vision tasks, complex planning, online learning and scheduling. Portia and insects in general are much more flexible to changes than our Neural nets. Changing objectives or a slight modification in the rules will often require retraining and a new design but insects can adapt to various lighting, environmental, prey, predator and navigational concerns. From the wiki on portia:

> Laboratory studies show that Portia learns very quickly how to overcome web-building spiders that neither it nor its ancestors would have met in the wild.

We should strive for its kind of energy and hence, computational efficiency in our models. Think for example, what that would mean for prosthetics. I find the overzealous defense of the current state of the art just as frustrating as out of hand dismissals of AI which constantly move goal posts.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#92

Post by YeGoblynQueenne »

gpderetta wrote: post #83
YeGoblynQueenne wrote: post #73

Earlier quoted context omitted.

>> how many generations of selective breeding do you think it would take to evolve a spider that could beat the best humans at go? That's a different issue. Spiders, as all insects, don't learn . It seems that insects have some kind of firmware-style programming that can't change, because they don't have the cognitive ability to change it. This firmware can be unbelievably complex, like in the case of Portia spiders…

> That's a different issue. Spiders, as all insects, don't learn. But portia spiders do seem to exhibit learning behavior. While they have preprogrammed strategies for common preys, they do attempt to catch unfamiliar preys and seem to learn from their mistakes and come up with new strategies.

That's planning, not learning.

An instance of learning would be training a spider to play Go, say.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#93

Post by xyzzy123 »

YeGoblynQueenne wrote: post #82
xyzzy123 wrote: post #63

Earlier quoted context omitted.

Spiders have much better hardware than the state of the art in low cost miniatuarised robotics. However I'm not convinced that if you trained a deep net and gave it access to a spider's sensor suite and effectors, that it would do worse than an actual spider would. A lot worse in terms of efficiency per watt, sure...

How would you train that deep net? What data would you give it? What would you train it to do exactly?

Survive. Reproduce. Just like the video games nets are already trained to play, there's a pretty clear scoring function. The inputs would be spider sensorium, the outputs would be spider effectors.

I'm kind of waiting for someone to do this already with C elegans by excising their 300 or so neurons and replacing them with a GPU and laser (but I imagine the practical impediments are way higher than I handwavingly imagine).

Re: Artificial Intelligence Software Is Booming, But Why Now?

#94

Post by YeGoblynQueenne »

Cybiote wrote: post #80
Houshalter wrote: post #76

Earlier quoted context omitted.

That's exactly my point. Spiders can't learn. Maybe they are amazing for what they do, but the comparison is apples to oranges. They do fundamentally different things. Of course spiders are complex. Everything in biology is complex. It's been slowly refined over millions and millions of years. Countless trillions of tweaks to its DNA tested and rejected until it hit upon something that worked. No human could design s…

The large majority of deployed machine learning algorithms actually can't learn. The bias is so large that most (all?) specialized hardware assumes that only inference will be done. In contrast, there is evidence for online adaptability in insects^ ^ http://www.annualreviews.org/eprint/5PHddkgYYKaduPp4CcN5/ful...

>> The large majority of deployed machine learning algorithms actually can't learn.

Learning algorithms construct a model of data- that's what we call "learning", or "training". The trained model is then used to make decisions, so that's what you would deploy in a production system.

In that sense the deployed systems don't learn any further, yes, but that's because you don't deploy the algorithm per se.

An exception is what is known as "online learning" algorithms, that can continuously update their model as new data becomes available (hance, "online").

Re: Artificial Intelligence Software Is Booming, But Why Now?

#95

Post by js8 »

vonnik wrote: post #32
thr0waway1239 wrote: post #9

TLDR from article which supplement's eli_gottlieb's comment: "Much of today’s A.I. boom goes back to 2006, when Amazon started selling cheap computing over the internet. Those measures built the public clouds of Amazon, Google, IBM and Microsoft, among others. That same year, Google and Yahoo released statistical methods for dealing with the unruly data of human behavior. In 2007, Apple released the first iPhone, a d…

AI is moving fast. Faster than it has in a long time. In research circles, "supervised learning", or data classification problems, are considered solved. Consider the magnitude of that statement: If we have sufficient labeled data, we can predict those labels accurately on data we haven't been exposed to: fraud, faces, diseases, you name it. AI is moving fast for three reasons, which Andrew Ng summarizes neatly: 1) W…

> It's more than fancy linear regression.

I am not an expert, but it seems to me that "big data algorithms" are actually simpler and require less ingenuity than classical statistics. Just look at a difference between frequentist and bayesian approaches in statistics. In frequentist methods, you need to choose model wisely, you're constrained what you can model, and there are many different models with different assumptions. With bayesian methods, you just represent any distribution somehow and do the calculation on those. Less assumptions for the models, more general models, but at the expense of more computations.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#96

Post by Teever »

YeGoblynQueenne wrote: post #92
gpderetta wrote: post #83

Earlier quoted context omitted.

> That's a different issue. Spiders, as all insects, don't learn. But portia spiders do seem to exhibit learning behavior. While they have preprogrammed strategies for common preys, they do attempt to catch unfamiliar preys and seem to learn from their mistakes and come up with new strategies.

That's planning, not learning. An instance of learning would be training a spider to play Go, say.

What is the difference between planning and learning, and would you consider a human that is unable to learn something like general relativity because they are mentally retarded to be unable to learn?

Re: Artificial Intelligence Software Is Booming, But Why Now?

#97

Post by Cybiote »

YeGoblynQueenne wrote: post #94
Cybiote wrote: post #80

Earlier quoted context omitted.

The large majority of deployed machine learning algorithms actually can't learn. The bias is so large that most (all?) specialized hardware assumes that only inference will be done. In contrast, there is evidence for online adaptability in insects^ ^ http://www.annualreviews.org/eprint/5PHddkgYYKaduPp4CcN5/ful...

>> The large majority of deployed machine learning algorithms actually can't learn. Learning algorithms construct a model of data- that's what we call "learning", or "training". The trained model is then used to make decisions, so that's what you would deploy in a production system. In that sense the deployed systems don't learn any further, yes, but that's because you don't deploy the algorithm per se. An exception…

I took care to point out deployed models. I did not say the models did no learning. In fact, one can draw an analogy from evolution and instinct to offline training/hyperparameter search/architecture tweaks and deployed model.

And, technically, the manner by which most nets are trained, via (minibatched) stochastic gradient descent is online. But the architectures are so ill-adapted to learning continuously and backprop is so data inefficient that retraining on new data is often just better done from scratch (or at best, freezing weights).

Re: Artificial Intelligence Software Is Booming, But Why Now?

#98

Post by YeGoblynQueenne »

xyzzy123 wrote: post #93
YeGoblynQueenne wrote: post #82

Earlier quoted context omitted.

How would you train that deep net? What data would you give it? What would you train it to do exactly?

Survive. Reproduce. Just like the video games nets are already trained to play, there's a pretty clear scoring function. The inputs would be spider sensorium, the outputs would be spider effectors. I'm kind of waiting for someone to do this already with C elegans by excising their 300 or so neurons and replacing them with a GPU and laser (but I imagine the practical impediments are way higher than I handwavingly imag…

>> The inputs would be spider sensorium, the outputs would be spider effectors.

That sounds great on paper. In practice, how do you even begin to collect "spider sensorium"? fMRI?

As to "spider effectors" I'm pretty sure we don't have the tech to do that yet, at least not in spider-scale (excluding Giant Spiders of Doom).

On paper, you can approximate any function with a multi-layer perceptron with enough layers. In practice, "enough" layers may be infinite and you'd need infinite amounts of data infinitely difficult to collect.

Re: Artificial Intelligence Software Is Booming, But Why Now?

#99

Post by dharma1 »

dragandj wrote: post #71
dharma1 wrote: post #67

Earlier quoted context omitted.

The key libraries part has kept me wondering. How much would it cost for AMD to assign a handful heavy duty engineers to this task (writing AMD optimised kernels for convolution etc)? Their management has been fast asleep for at least 2 years

That is a part of the problem: they assigned people for the task, and produced open-source libraries for matrices, FFT, maybe even something for DNNs. But, those are not polished much, and you have to hunt them down and install them yourself. And, they do a really bad job at marketing. On the other hand, finding and installing those libraries is nothing compared to actually developing GPU computing software, so, as I…

I haven't seen CuDNN equivalents (in terms of perf) for common machine learning frameworks from AMD. I don't think they exist, if they did, people would shift to using AMD.

For NVidia I have seen some faster kernels than the ones supplied by NVidia - https://github.com/NervanaSystems/neon - though CuDNN introduced Winograd kernels too in their last update

Re: Artificial Intelligence Software Is Booming, But Why Now?

#100

Post by xyzzy123 »

YeGoblynQueenne wrote: post #98
xyzzy123 wrote: post #93

Earlier quoted context omitted.

Survive. Reproduce. Just like the video games nets are already trained to play, there's a pretty clear scoring function. The inputs would be spider sensorium, the outputs would be spider effectors. I'm kind of waiting for someone to do this already with C elegans by excising their 300 or so neurons and replacing them with a GPU and laser (but I imagine the practical impediments are way higher than I handwavingly imag…

>> The inputs would be spider sensorium, the outputs would be spider effectors. That sounds great on paper. In practice, how do you even begin to collect "spider sensorium"? fMRI? As to "spider effectors" I'm pretty sure we don't have the tech to do that yet, at least not in spider-scale (excluding Giant Spiders of Doom). On paper, you can approximate any function with a multi-layer perceptron with enough layers. In…

I think the most reasonable near-term approach "if you had to do it" would be reverse engineering and virtualities, but please bear in mind it was a hypothetical to begin with.

I expect the number of inputs to spider sensorium would be low thousands of analog inputs. To be clear I can't cite a reference for this.

Yes it seems intractibly difficult on paper but spider hardware can do it.

It's hard to rate what we'll expect from self driving cars on the flatworm to spider spectrum (probably closer to the flatworm end, but the cameras add an interesting dimension), but clearly reasonable yet ambitious people believe these problems are tractable with sufficient effort.

Post reply on HN