Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RCN is much more data efficient than traditional Deep Neural Networks

vicarious.com

Post reply on HN
Print view View original

11–20 of 56 posts

Re: RCN is much more data efficient than traditional Deep Neural Networks

#11

Post by dpandya »

It seems that the primary contribution of this technique is that it uses specific assumptions supported by neuroscience research in order to allow for composability of learning and better generalization. By introducing these specific assumptions (e.g. contours define objects), they are able to reduce the complexity that the model has to learn and thereby reduce the amount of data that it needs.

Obviously, the question then becomes: what happens when you have visual situations that violate or come close to violating the assumptions made?

I'm not familiar enough with the specifics of RCNs to be able to answer this; maybe someone else can. Very interesting paper and approach regardless.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#12

Post by cwilkes »

BucketSort wrote: post #4

I'd love to read this, but the faint text on white background... good god. I went through the code looking to change the background so I could read it and found this: body { text-rendering: optimizeLegibility; } Ok

If you are on an iPhone (maybe any Safari browser?) clicking on the top left reader button makes it legible.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#13

Post by cs702 »

This paper looks really interesting to me, although after quickly reading the introduction it's evident that I'm going to have to invest quite a bit of time and effort on the paper to grasp its key ideas. I come from more an encoding-decoding, deep/machine-learning background, as opposed to a probabilistic graphical modeling or PGM background, and my knowledge of neuroscience is minimal.

To date, my experience with "deep PGM models" (for lack of a better term) is limited to some tinkering with (a) variational autoencoders using ELBO maximization as the training objective, and to a much lesser extent (b) "bi-directional" GANs using a Jensen-Shannon divergence between two joint distributions as the training loss.

Has anyone here with a similar background to mine had a chance to read this paper? Any thoughts?

Re: RCN is much more data efficient than traditional Deep Neural Networks

#14

Post by boltzmannbrain »

flor1s wrote: post #9

I only skimmed over the article, but I think the title on HN does not reflect the claims the authors are making. The title of the paper is: A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs The title of the article is: Common Sense, Cortex, and CAPTCHA That's nowhere near the sensationalist title at HN: RCN is much more data efficient than traditional Deep Neural Networks

Paper abstract highlights the model's data efficiency several times:

Learning from few examples and generalizing to dramatically different situations are capabilities of human visual intelligence that are yet to be matched by leading machine learning models. By drawing inspiration from systems neuroscience, we introduce a probabilistic generative model for vision in which message-passing based inference handles recognition, segmentation and reasoning in a unified way. The model demonstrates excellent generalization and occlusion-reasoning capabilities, and outperforms deep neural networks on a challenging scene text recognition benchmark while being 300-fold more data efficient. In addition, the model fundamentally breaks the defense of modern text-based CAPTCHAs by generatively segmenting characters without CAPTCHA-specific heuristics. Our model emphasizes aspects like data efficiency and compositionality that may be important in the path toward general artificial intelligence.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#15

Post by [deleted] »

cwilkes wrote: post #12
BucketSort wrote: post #4

I'd love to read this, but the faint text on white background... good god. I went through the code looking to change the background so I could read it and found this: body { text-rendering: optimizeLegibility; } Ok

If you are on an iPhone (maybe any Safari browser?) clicking on the top left reader button makes it legible.

[deleted]

Re: RCN is much more data efficient than traditional Deep Neural Networks

#18

Post by sherbondy »

As far as I can tell, the code on GitHub (https://github.com/vicariousinc/science_rcn) only works for the MNIST dataset.

Unclear how to run on the CAPTCHA examples referenced in the paper, even though they did make the datasets for those examples available.

Bummer, a big part of what the paper mentions about being so great with this RCN model is being able to segment sequences of characters (of indeterminate length even!). Sad that I cannot easily verify this for myself!

Re: RCN is much more data efficient than traditional Deep Neural Networks

#20

Post by fuelfive »

sherbondy wrote: post #18

As far as I can tell, the code on GitHub ( https://github.com/vicariousinc/science_rcn ) only works for the MNIST dataset. Unclear how to run on the CAPTCHA examples referenced in the paper, even though they did make the datasets for those examples available. Bummer, a big part of what the paper mentions about being so great with this RCN model is being able to segment sequences of characters (of indeterminate length…

We talked about releasing more comprehensive proof of concept code, but ultimately decided against it. While helpful for other researchers, offering anyone on the internet a ready-to-use arbitrary captcha breaker seemed like a net-negative for society.
Post reply on HN