Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RCN is much more data efficient than traditional Deep Neural Networks

vicarious.com

Post reply on HN
Print view View original

21–30 of 56 posts

Re: RCN is much more data efficient than traditional Deep Neural Networks

#21

Post by joe_the_user »

dpandya wrote: post #11

It seems that the primary contribution of this technique is that it uses specific assumptions supported by neuroscience research in order to allow for composability of learning and better generalization. By introducing these specific assumptions (e.g. contours define objects), they are able to reduce the complexity that the model has to learn and thereby reduce the amount of data that it needs. Obviously, the questio…

After six or seven click-throughs, I downloaded the PDF.

I haven't read it but skimming, I could see that there definitely were no formulas in it at all . Which sort of says, at best what it tells you is "we did this thing, which is kind of like X and kind of like Y with Z changes". Essentially, no way to reproduce or understand by itself. The first reference then had a link behind a paywall...

So despite lots of apparent explanation, it seems like what they're actually doing is essentially unspecified (at least to the interested layman). It seems like at best an expert in the field of "compositional models" could say what is happening.

Also, the paper is published under the heading of an AI firm Fremont, ca rather than folks in a university, with the many authors listed by initial and last name...

PDF for the curious:

http://science.sciencemag.org/content/sci/early/2017/10/26/s...

Edit: tracked down that apparently has some "real" math. Whether is even what the OP is doing remains to be seen.

https://staff.fnwi.uva.nl/t.e.j.mensink/zsl2016/zslpubs/lake...

Re: RCN is much more data efficient than traditional Deep Neural Networks

#23

Post by boltzmannbrain »

joe_the_user wrote: post #21
dpandya wrote: post #11

It seems that the primary contribution of this technique is that it uses specific assumptions supported by neuroscience research in order to allow for composability of learning and better generalization. By introducing these specific assumptions (e.g. contours define objects), they are able to reduce the complexity that the model has to learn and thereby reduce the amount of data that it needs. Obviously, the questio…

After six or seven click-throughs, I downloaded the PDF. I haven't read it but skimming, I could see that there definitely were no formulas in it at all . Which sort of says, at best what it tells you is "we did this thing, which is kind of like X and kind of like Y with Z changes". Essentially, no way to reproduce or understand by itself. The first reference then had a link behind a paywall... So despite lots of app…

70 page supplementary material: http://science.sciencemag.org/content/sci/suppl/2017/10/25/s...

Reference code: https://github.com/vicariousinc/science_rcn

Re: RCN is much more data efficient than traditional Deep Neural Networks

#24

Post by [deleted] »

joe_the_user wrote: post #21
dpandya wrote: post #11

It seems that the primary contribution of this technique is that it uses specific assumptions supported by neuroscience research in order to allow for composability of learning and better generalization. By introducing these specific assumptions (e.g. contours define objects), they are able to reduce the complexity that the model has to learn and thereby reduce the amount of data that it needs. Obviously, the questio…

After six or seven click-throughs, I downloaded the PDF. I haven't read it but skimming, I could see that there definitely were no formulas in it at all . Which sort of says, at best what it tells you is "we did this thing, which is kind of like X and kind of like Y with Z changes". Essentially, no way to reproduce or understand by itself. The first reference then had a link behind a paywall... So despite lots of app…

[deleted]

Re: RCN is much more data efficient than traditional Deep Neural Networks

#26

Post by [deleted] »

[deleted] wrote: post #25
gugagore wrote: post #22

Here's another example of a generative model that improves data efficiency, in a similar-ish domain. https://gizmodo.com/a-new-ai-system-passed-a-visual-turing-t... / http://web.mit.edu/cocosci/Papers/Science-2015-Lake-1332-8.p...

[deleted]

[deleted]

Re: RCN is much more data efficient than traditional Deep Neural Networks

#28

Post by stochastic_monk »

If I'm not mistaken, a Deep Belief Net or Deep Belief Machine would also be a generative model with enormously greater data efficiency. Comparing against CNNs is a red herring: the advantage of requiring less data to develop a model is more a generative/discriminative issue than it is an "RCN vs everyone else" issue.

What I don't quite understand is why Deep Belief Nets seem to not be getting press these days. For example, see this paper from 2010: http://proceedings.mlr.press/v9/salakhutdinov10a.html.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#29

Post by Groxx »

BucketSort wrote: post #4

I'd love to read this, but the faint text on white background... good god. I went through the code looking to change the background so I could read it and found this: body { text-rendering: optimizeLegibility; } Ok

Huh. Did they change it? I see a very thin font in the header and in bulleted lists, but the rest of the text on the page is black (literally #000000) and relatively bold compared to what I'm used to seeing online (could just be that it's slightly larger, which is also good! it's by no means big, just nice to see something not pointlessly tiny).

The header has the awful "ObjektivMk1-Thin" font mentioned elsewhere, but for me the body is a normal "Roboto","Helvetica Neue",Helvetica,Arial,sans-serif font-family.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#30

Post by [deleted] »

stochastic_monk wrote: post #28

If I'm not mistaken, a Deep Belief Net or Deep Belief Machine would also be a generative model with enormously greater data efficiency. Comparing against CNNs is a red herring: the advantage of requiring less data to develop a model is more a generative/discriminative issue than it is an "RCN vs everyone else" issue. What I don't quite understand is why Deep Belief Nets seem to not be getting press these days. For ex…

[deleted]
Post reply on HN