Really wasn't expecting a Terry Davis comment at the bottom.
Who is Terry Davis.
I didn't know either, but found this: http://motherboard.vice.com/read/gods-lonely-programmer
Scary and fascinating.
Suddenly, a leopard print sofa appears
51–60 of 114 posts
Re: Suddenly, a leopard print sofa appears
#52> So I guess, there's still a lot of work to be done. And I think this is the most interesting part. One of the most depressing things about all of the "this image recognition algorithm performs better than humans on this task" is the idea that we've pretty much solved the problem, and it's just a matter of some more optimization and tweaking to handle a few edge cases. This kind of problem, where the dominant soluti…
Definitely agree here, want to add a link to this paper which shows how far we still have to go (and questions whether the current models will ever replicate human vision): http://arxiv.org/abs/1412.1897
Re: Suddenly, a leopard print sofa appears
#53Leopards (or jaguars) are complex 3-dimensional shapes with quite a lot of degrees of freedom (considering all the body parts that can move independently). These shapes can produce a lot of different 2d contours My son keeps telling me that infants are fine with, say, a truck transforming into a clown (when it emerges from the other side of a visual barrier) but not with it transforming into TWO of something. Apparen…
> but not with it transforming into TWO of something
I suppose that's why Banach–Tarski is considered a paradox.
Re: Suddenly, a leopard print sofa appears
#54> the problem won't be solved by collecting even larger datasets and using more GPUs, because leopard print sofas are inevitable. The models have room for improvement, but it's not clear to me that larger datasets won't solve the problem. Larger datasets and more processing power is exactly why neural nets have surged in effectiveness recently. Who knows how much further current models can go with more data and proce…
read the Hinton paper cited in the OP -- no amount of processing power will make CNNs represent structured latent variables
Re: Suddenly, a leopard print sofa appears
#55This is the best HN submission I've seen in a very long time. Really thought-provoking.
I agree. I don't work with machine learning or neural nets, and thus don't have more than the most cursory laymans understanding of them. This article read really well and was quite informative of the problems this technology is facing.
Re: Suddenly, a leopard print sofa appears
#56This article would not come as a surprise to anyone who works with ConvNets. Sadly, that might not the case for those outside of the field, largely due to media's inadequate coverage of our advances (but this is common outside our field too). No one in the field really believes ConvNets see better than humans. They are very good single glance texture recognizers. It's as if you flashed an image and looked at it for a…
Next step is Video. Adding temporal dimension will emphasize extrapolating true 3d shapes of recognized objects.
Re: Suddenly, a leopard print sofa appears
#57Google image search says
Best guess for this image: cat bed furniture
Re: Suddenly, a leopard print sofa appears
#58ConvNets have gotten popular because of their strong empirical results. All the recent work on visualizing CNNs suggests that the community working on Deep Learning still has a lot to learn about their own algorithms. But high-level notions like a Jaguar is a cat-like animal aren't necessary to perform well on an N-way classification task like ImageNet. What's more important to note is everybody knows there's plenty…
I doubt that reviewers for NIPS would reject a paper with a novel approach because it didn't perform at best in class level, provided it offered a way forward.
If it doesn't work at all, or isn't a new idea, that's different.
Re: Suddenly, a leopard print sofa appears
#59Suddenly, a shaded rock appears. https://upload.wikimedia.org/wikipedia/commons/7/77/Martian_... We're doing humans wrong. Maybe not all wrong, and of course, humans are extremely useful things, but think about it: sometimes it almost looks like we're already there. There always going to be an anomaly; lots of them, actually, considering all the things shaded in different patterns. Something have to change. I agree t…
> We're doing humans wrong.
I don't follow. If you asked a human what the linked image looked like, they'd likely say a face, but if you then asked them what it actually was, they're all going to change their answer to a rock, even specifically a rock on Mars (if given a colour version of this image).
It's true that humans see patterns that aren't there, but does that detract from our ability to recognise objects?
Re: Suddenly, a leopard print sofa appears
#60Thank you for this article; very thought provoking.
My nitpick:
> When each student was given a heavy book of MNIST database, hundreds of pages filled with endless hand-written digit series, 60000 total, written in different styles, bold or italic, distinctly or sketchy. > ... > So, are you going to say that was not the case?
I understand the point the author is making. Human brains are really good at taking limited examples and correctly extrapolating them to new cases. That is, of course, the goal of intelligence. Machine Learning has gotten better at this generalization, but has a long way to go. And ConvNets as they exist today will not achieve that, no matter how much training you perform on them.
This specific example is inaccurate though. Let us aggressively simplify and low-ball by saying that humans see at 24fps. Humans of course don't see in discrete frames, but this simplification doesn't detract from my argument and makes quantifying easier. So, if you give a human a single page of numbers, and they look at it for an hour, they have now seen >86k examples. That's 86k examples with twitching saccades, and from both eyes. That's in just an hour of looking at numbers.
Prior to being given that page of numbers, most children will have been alive for 4-5 years. That's 3 billion examples from a wide variety of subjects (we ignore sleeping cycles, because we're already low-balling this fps figure, and because the brain is still learning and visualizing during sleep).
And humans are born with a pre-built visual cortex. Edge detection, gradient detection, etc. are all already built for us. CNNs learn that from scratch.
The author's real point is still valid, though, don't get me wrong. I'm just nitpicking.