Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RCN is much more data efficient than traditional Deep Neural Networks

vicarious.com

Post reply on HN
Print view View original

31–40 of 56 posts

Re: RCN is much more data efficient than traditional Deep Neural Networks

#31

Post by visarga »

fuelfive wrote: post #20
sherbondy wrote: post #18

As far as I can tell, the code on GitHub ( https://github.com/vicariousinc/science_rcn ) only works for the MNIST dataset. Unclear how to run on the CAPTCHA examples referenced in the paper, even though they did make the datasets for those examples available. Bummer, a big part of what the paper mentions about being so great with this RCN model is being able to segment sequences of characters (of indeterminate length…

We talked about releasing more comprehensive proof of concept code, but ultimately decided against it. While helpful for other researchers, offering anyone on the internet a ready-to-use arbitrary captcha breaker seemed like a net-negative for society.

Like people can't find captcha breakers already. Your research might push for the replacement of text captchas with some other test.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#32

Post by [deleted] »

dpandya wrote: post #11

It seems that the primary contribution of this technique is that it uses specific assumptions supported by neuroscience research in order to allow for composability of learning and better generalization. By introducing these specific assumptions (e.g. contours define objects), they are able to reduce the complexity that the model has to learn and thereby reduce the amount of data that it needs. Obviously, the questio…

[deleted]

Re: RCN is much more data efficient than traditional Deep Neural Networks

#33

Post by visarga »

This is a paper that departs from the 'normal' AI routine and takes a very different approach. Is there another paper formally describing the RCN network? What goes inside the RCN cell? I find it more like a teaser than a revelation at this point.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#35

Post by dpandya »

visarga wrote: post #33

This is a paper that departs from the 'normal' AI routine and takes a very different approach. Is there another paper formally describing the RCN network? What goes inside the RCN cell? I find it more like a teaser than a revelation at this point.

The details are provided in the supplementary material: http://science.sciencemag.org/content/sci/suppl/2017/10/25/s...

(mentioned by boltzmannbrain in one of the other comments)

Re: RCN is much more data efficient than traditional Deep Neural Networks

#36

Post by bufo »

Again: no one cares about CAPTCHA in the deep learning world compared to other more challenging benchmarks. I wouldn’t be surprised that many optimizations could be made with ANY kind of effort put into it. Still waiting for Vicarious to go beyond MNIST and text CPATCHA.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#37

Post by barbolo »

cs702 wrote: post #13

This paper looks really interesting to me, although after quickly reading the introduction it's evident that I'm going to have to invest quite a bit of time and effort on the paper to grasp its key ideas. I come from more an encoding-decoding, deep/machine-learning background, as opposed to a probabilistic graphical modeling or PGM background, and my knowledge of neuroscience is minimal. To date, my experience with "…

I’m reading over and over again since the last weekend. And I’m checking the code. And I’m still not understanding it.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#38

Post by nl »

bufo wrote: post #36

Again: no one cares about CAPTCHA in the deep learning world compared to other more challenging benchmarks. I wouldn’t be surprised that many optimizations could be made with ANY kind of effort put into it. Still waiting for Vicarious to go beyond MNIST and text CPATCHA.

This is trueish, but there is more to it than that.

It is true for sure that absolute performance on MNIST isn't the most interesting thing in the world.

But when introducing a new tool or technique being able to show competitive performance on MNIST is a good way to show that it isn't an entirely useless thing.

I'd note that recent Sabour, Frosst and Hinton paper[1] (where they finally got Hinton's capsules to work) spends most of the paper analyzing how it performs on MNIST, and only a short section on other datasets.

I assume I don't need to point out that Geoff Hinton does know a little about deep learning, and if he thinks submitting a NIPS paper on MNIST is acceptable in 2017 then I'm not going to argue too hard against it.

[1] https://arxiv.org/pdf/1710.09829.pdf

Re: RCN is much more data efficient than traditional Deep Neural Networks

#39

Post by chronice70 »

nl wrote: post #38
bufo wrote: post #36

Again: no one cares about CAPTCHA in the deep learning world compared to other more challenging benchmarks. I wouldn’t be surprised that many optimizations could be made with ANY kind of effort put into it. Still waiting for Vicarious to go beyond MNIST and text CPATCHA.

This is trueish, but there is more to it than that. It is true for sure that absolute performance on MNIST isn't the most interesting thing in the world. But when introducing a new tool or technique being able to show competitive performance on MNIST is a good way to show that it isn't an entirely useless thing. I'd note that recent Sabour, Frosst and Hinton paper[1] (where they finally got Hinton's capsules to work)…

And what about the other boys who know a thing or two about deep learning? I don't see any of these people submitting MNIST to NIPS in 2017: Yousha Bengio, Yann LeCun, Ian Goodfellow, Andrew Ng, Ross Girshick, Andrej Karpathy, Pedro Domingos, and the whole DeepMind crew.

So yes, submitting experiments on MNIST in 2017 should not be taken seriously.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#40

Post by _0w8t »

I do not see a discussion in the paper regarding computational efficiency of RCN detection. The only hint about performance that I found is at the end of supplementary material where the authors state:

> Use of appearance during the forward pass: Surface appearance is now only used after the backward pass. This means that appearance information (including textures) is not being used during the forward pass to improve detection (whereas CNNs do). Propagating appearance bottom-up is a requisite for high performance on appearance-rich images.

I presume from this that in the current form RCN requires much more computations than CNN per detection, but I could be wrong.

Post reply on HN