Didn't read article responding purely to the title: AI is a three-trick pony -Regression -Classification -Clustering Nothing more and nothing less.
AI is not just the contents of your undergrad machine learning class.
Is AI Riding a One-Trick Pony?
151–160 of 219 posts
Re: Is AI Riding a One-Trick Pony?
#152Earlier quoted context omitted.
OTOH, we are merely at circa Year Five into deep reinforcement learning research. It started as a cluster of 16M CPUs having taught itself to recognize a cat 95% of the time after training on 1B google images. And we are now at One-Shot Imitation Learning, "a general system that can turn any demonstrations into robust policies that can accomplish an overwhelming variety of tasks". One Shot Imitation Learning https://…
Not really, no. Saying we are at year five of Deep RL is about as informative as saying we are at year five of deep learning. Reinforcement learning as a field goes back decades.
But now we have GPUs, which makes it entirely different. /s
And it kinda does, but in an engineering way rather than a statistics way.
Like reinforcement learning from pixels is pretty new (i would be really interested if you have 10+year old citations), and pretty amazing. I've been looking at RL (through OpenAI gym) and realising that I "just" need to annotate a bunch of images and then train a network that will predict (fire/no fire in Doom from those pixels, and I can just add another network that builds some history onto this net (like an RNN) and this might actually work, is kinda amazing.
I'm still not sure I believe that it's always a good approach, but some of my initial experiments with my own (mostly image so far) data have been pretty promising.
The hype is pretty annoying though, especially if you've been interested in these things for years.
The bar to entry for these kinds of applications has been significantly lowered, which means we'll see more of it. I guess, in some sense, it's similar to the explosion of computer programs following the advent of personal computers (maybe, I haven't thought deeply about this part).
Re: Is AI Riding a One-Trick Pony?
#153Earlier quoted context omitted.
> 1) learn how the brain works 2) build a simulator I disagree that step #1 is important. Consider the "Air-foil", which led to flight. In one sense, its an approximation of the wings of birds and other animals. But ultimately, the discovery that the "Air-foil" shape turns sideways blowing wind into an upward force now called "lift" is completely different from how most people understand bird wings. Bird Wings flap,…
Disclaimer: I have no expertise in AI. That said, I agree that learning how the brain works seems unimportant and unnecessary. Evolution doesn't know how a brain works, but it's given us Einstein, Michelangelo, and conversations on HN. It seems really important to learn how to build evolution into attempts at AI, given that evolution is the only known mechanism that leads to what we recognize as intelligence.
>evoluyion doesn't
you use antropomorphy to reflect on your own standpoint. we don't know how the brain works? we can feel it and psychologist have a huge body of work concerned with the topic and that is already having influence on competition and fitness.
Re: Is AI Riding a One-Trick Pony?
#154The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…
Hinton was able to survive 30 years in the academic wilderness. Most academics can't. Thus they work on "safe" projects.
It can be done, even today. If you work outside the US and work on cheap things (i.e. no special equipment), especially if you can teach then you can hang around for a long time.
I have met a lot of academics like this over the years, but I think your broader point might be that this is not possible today, which I agree with, and which is why I left academia (modulo personal situations).
Re: Is AI Riding a One-Trick Pony?
#155I'd argue that the next problem to attack is manipulation in unstructured environments. Robots suck at that. There's been amazingly little progress in the last 40 years. DARPA had a manipulation project and the DARPA humanoid challenge a few years ago, and they got as far as key-in-lock and throwing a switch. Amazon is still trying to get general bin-picking to work. Nobody has fully automatic sewing that works well, except the people who starch the fabric and make it temporarily rigid. Willow Garage got towel-folding to work, but general laundry folding was beyond them. This is embarrassing.
Many of the mammals can do this, down to the squirrel level. It doesn't take human-level AI. There are rodents with peanut-size brains that can do it.
It's a well-defined problem, measuring success is easy, it doesn't take that much hardware, and has a clear payoff. We just have no clue how to do it.
Re: Is AI Riding a One-Trick Pony?
#156Earlier quoted context omitted.
It's not that we need to understand our neural networks better, it's that we need to understand our problem domain better. How 'bout "creating models that can work with more dimensions of the problem domain than are conveyed by standard data labeling"? I mean, we don't simply want AI but actually "need" it in the sense that problems like biological system are too complex to understand without artificial enhancements…
It will probably play out like a conversation. A data scientist trains an ML model, and in analyzing the results discovers some intrinsic property or invariant of the problem domain. The scientist can then encode that information into the model and retrain. And that goes on and on, each time providing more accurate results. As an aside, I think it's important that we find a way to examine and inspect how an ML model…
This is somewhat practical for neural networks. For example, instead of minimizing the loss function, why not tweak the input to maximize a neuron’s activation? Or with a CNN, maximize the sum of a kernel’s channel? This would tell us what the neuron corresponds with. This is what Google did with DeepDream.
An explanation/tutorial, with clean images of the process: https://github.com/tensorflow/tensorflow/blob/r0.10/tensorfl...
Google’s investigation of it’s GoogLeNet architecture: http://storage.googleapis.com/deepdream/visualz/tensorflow_i...
Now, I say somewhat because results can be visually confusing, ex Google’s analysis. Even then, we can see the progression of layer complexity as we go deeper into ImageNet. Plus, we can see mixed4b_5x5_bottleneck_pre_relu has kernels that seem to correspond with noses and eyes. mixed_4d_5x5_pre_relu has a kernel that seems to correspond with cat faces.
Re: Is AI Riding a One-Trick Pony?
#157The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…
Hinton's quote is taken a bit out of context though. I just watched his interview on Andrew Ng's "Neural Networks and Deep Learning" class on Coursera and he seemed convinced that the next "breakthrough" will come from (a variant on) neural networks.
Right but to extend the bridge analogy what's interesting isn't the materials (neural nets) as such but the structure and why that structure works.
Re: Is AI Riding a One-Trick Pony?
#158There is a bit of "can't see the forest for the trees" failure in the article. AI is spearheading a paradigm shift in how we write programs. Or rather, we don't write programs. We write much much shorter programs that search the program space for programs that satisfy some desiderata. The programs we get as the output of the search process are extremely flexible, work very well, are very homogeneous in compute (e.g.…
That is a good way to think about it. I am really encourage by new non-sequential architectures, multiple input and output channels, etc., and how these architectures can be expressed in functional Keras. It seems like building with something like functional composition is another paradigm shift. BTW, thanks for your writing, especially about RNNs: really useful when the ordering of sequential data is important. I use what I have learned from reading your blogs and papers literally every day at work.
Re: Is AI Riding a One-Trick Pony?
#159Earlier quoted context omitted.
Human perception and intelligence are by no means infallible, but neither are they anywhere near being as brittle as current AI. The thing about illusions is that we generally know that we are being subject to an illusion, and we also usually have the depth of understanding to know when we don't understand something about what we are seeing or think we have heard, and we have the depth of understanding to think of ac…
> The thing about illusions is that we generally know that we are being subject to an illusion. Citation needed? I did my undergrad in cognitive science, and while my knowledge of illusions is very limited, I never came across anything to suggest that we have an innate awareness of when our perceptual system is being tricked.
Wait, isn't that completely obvious? As a kid I used to seek them out precisely for the thrill of feeling my brain being tricked. Nobody needs to be told why MC Escher drawings are fun to look at.
Re: Is AI Riding a One-Trick Pony?
#160Yes there are still lots of people trying to take Hinton and other's old approach and rediscovering the many ways it lacks for general intelligence. However, it is also the case that people are making a shitton of progress in overcoming those problems, both while keeping some of those old DL assumptions and by discarding many of them.
The pessimistic articles never seem to be aware of research like these: https://arxiv.org/abs/1612.00796 and https://hackernoon.com/feynman-machine-a-new-approach-for-co... .