Live data from Hacker News

RCN is much more data efficient than traditional Deep Neural Networks

vicarious.com

51–56 of 56 posts

Re: RCN is much more data efficient than traditional Deep Neural Networks

#51

Earlier quoted context omitted.

After six or seven click-throughs, I downloaded the PDF. I haven't read it but skimming, I could see that there definitely were no formulas in it at all . Which sort of says, at best what it tells you is "we did this thing, which is kind of like X and kind of like Y with Z changes". Essentially, no way to reproduce or understand by itself. The first reference then had a link behind a paywall... So despite lots of app…

70 page supplementary material: http://science.sciencemag.org/content/sci/suppl/2017/10/25/s... Reference code: https://github.com/vicariousinc/science_rcn

I still find it incredibly hard to tell whether this is overblown hype or legit scientific progress. There is no indication whatsoever that this approach scales to deep feature hierarchies and that is likely what you need to compete on hard tasks like classification on ImageNet. Given the amount of money at play (several hundred millions of dollars), writing 70 pages, making code publishable is certainly an obvious way to get the most out of the hype.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#52
post #45

Earlier quoted context omitted.

Those are letters?

I had no issue recognising any of those immediately.

I recognized most of them, but only because I knew what they were supposed to be. I don't think I would have otherwise.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#53
post #29

I'd love to read this, but the faint text on white background... good god. I went through the code looking to change the background so I could read it and found this: body { text-rendering: optimizeLegibility; } Ok

Huh. Did they change it? I see a very thin font in the header and in bulleted lists, but the rest of the text on the page is black (literally #000000) and relatively bold compared to what I'm used to seeing online (could just be that it's slightly larger, which is also good! it's by no means big , just nice to see something not pointlessly tiny). The header has the awful "ObjektivMk1-Thin" font mentioned elsewhere, b…

They did change it.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#54

I'd love to read this, but the faint text on white background... good god. I went through the code looking to change the background so I could read it and found this: body { text-rendering: optimizeLegibility; } Ok

You're a datapoint in the upcoming Science paper, "RCN is Much Better at Reading Our Announcements Concerning RCN Than Actual Humans."

lol

Re: RCN is much more data efficient than traditional Deep Neural Networks

#55
It looks RCN sits between traditional machine learning (with manual feature selection) and 'modern' neural networks (CNN). The traditional methods are too rigid to capture the essential information, while the CNNs sometimes are too flexible to avoid overfitting. Different from CNNS, RCNs have a predetermined structure. Humans are not born a blank slate, we have a neural structure encoded in our genes, so we don't need millions of training samples to recognize objects. So maybe RCN is onto something.

I am curious how RCN performs on real-life images like ImageNet, and how do they perform against adversarial examples. If they can easily recognize adversarial examples, that would be very interesting...

Re: RCN is much more data efficient than traditional Deep Neural Networks

#56
post #38
post #36

Again: no one cares about CAPTCHA in the deep learning world compared to other more challenging benchmarks. I wouldn’t be surprised that many optimizations could be made with ANY kind of effort put into it. Still waiting for Vicarious to go beyond MNIST and text CPATCHA.

This is trueish, but there is more to it than that. It is true for sure that absolute performance on MNIST isn't the most interesting thing in the world. But when introducing a new tool or technique being able to show competitive performance on MNIST is a good way to show that it isn't an entirely useless thing. I'd note that recent Sabour, Frosst and Hinton paper[1] (where they finally got Hinton's capsules to work)…

+1. I was thinking about the Hinton Capsule paper using MNIST.
Post reply on HN