Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RCN is much more data efficient than traditional Deep Neural Networks

vicarious.com

Post reply on HN
Print view View original

41–50 of 56 posts

Re: RCN is much more data efficient than traditional Deep Neural Networks

#41

Post by singularity2001 »

bufo wrote: post #36

Again: no one cares about CAPTCHA in the deep learning world compared to other more challenging benchmarks. I wouldn’t be surprised that many optimizations could be made with ANY kind of effort put into it. Still waiting for Vicarious to go beyond MNIST and text CPATCHA.

one of the central points of the blog post was that the problem of CAPTCHAs / general artificial intelligence is NOT solved until letters like these are recognised: https://www.vicarious.com/wp-content/uploads/2017/10/image20...

does your network solve/recognise those?

Re: RCN is much more data efficient than traditional Deep Neural Networks

#42

Post by nl »

chronice70 wrote: post #39
nl wrote: post #38

Earlier quoted context omitted.

This is trueish, but there is more to it than that. It is true for sure that absolute performance on MNIST isn't the most interesting thing in the world. But when introducing a new tool or technique being able to show competitive performance on MNIST is a good way to show that it isn't an entirely useless thing. I'd note that recent Sabour, Frosst and Hinton paper[1] (where they finally got Hinton's capsules to work)…

And what about the other boys who know a thing or two about deep learning? I don't see any of these people submitting MNIST to NIPS in 2017: Yousha Bengio, Yann LeCun, Ian Goodfellow, Andrew Ng, Ross Girshick, Andrej Karpathy, Pedro Domingos, and the whole DeepMind crew. So yes, submitting experiments on MNIST in 2017 should not be taken seriously.

"boys"

Not sure what this was supposed to mean? Yes, I think Fei Fei Li's datasets are much better tests than MNIST if that is what you were getting at?

I don't see any of these people submitting MNIST to NIPS in 2017

None of them submitted things as entirely new and different as this, either.

Having said that, I think my point holds.

The completely awesome 2017 "Generalization in Deep Learning" paper[1] was co-authored by Bengio and uses MNIST - because everyone can follow it.

Yann LeCun was co-author on the 2017 "Adversarially Regularized Autoencoders for Generating Discrete Structures"[1.5], using MNIST

Ian Goodfellow Autoencoder NIPS paper[1] used MNIST as one of its 4 datasets. Yes, it was 2014, but when introducing a new technique using familiar datasets isn't a bad thing.

DeepMind's "Bayes by Backprop" (ICML15) used MNIST[2]

Another example: the (June 2017) John Langford (Vowpal Wabbit) et. al paper[3] on using Boosting to learn ResNet blocks used MNIST.

So yes, I agree there are much better datasets to compare performance on. But to prove something new works, MNIST is a useful dataset.

[0] https://arxiv.org/pdf/1710.05468.pdf

[1] http://papers.nips.cc/paper/5423-generative-adversarial-nets

[1.5] https://arxiv.org/pdf/1706.04223.pdf

[2] https://deepmind.com/research/publications/weight-uncertaint...

[3] https://arxiv.org/pdf/1706.04964.pdf

Re: RCN is much more data efficient than traditional Deep Neural Networks

#44

Post by Filligree »

singularity2001 wrote: post #41
bufo wrote: post #36

Again: no one cares about CAPTCHA in the deep learning world compared to other more challenging benchmarks. I wouldn’t be surprised that many optimizations could be made with ANY kind of effort put into it. Still waiting for Vicarious to go beyond MNIST and text CPATCHA.

one of the central points of the blog post was that the problem of CAPTCHAs / general artificial intelligence is NOT solved until letters like these are recognised: https://www.vicarious.com/wp-content/uploads/2017/10/image20... does your network solve/recognise those?

Those are letters?

Re: RCN is much more data efficient than traditional Deep Neural Networks

#45

Post by dx034 »

Filligree wrote: post #44
singularity2001 wrote: post #41

Earlier quoted context omitted.

one of the central points of the blog post was that the problem of CAPTCHAs / general artificial intelligence is NOT solved until letters like these are recognised: https://www.vicarious.com/wp-content/uploads/2017/10/image20... does your network solve/recognise those?

Those are letters?

I had no issue recognising any of those immediately.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#46

Post by dx034 »

> In 2013, we announced an early success of RCN: its ability to break text-based CAPTCHAs like those illustrated below (left column). With one model, we achieve an accuracy rate of 66.6% on reCAPTCHAs, 64.4% on BotDetect, 57.4% on Yahoo, and 57.1% on PayPal, all significantly above the 1% rate at which CAPTCHAs are considered ineffective (see [4] for more details). When we optimize a single model for a specific style, we can achieve up to 90% accuracy.

66% with reCaptcha and up to 90% when optimised is much higher than what I can achieve with my actual brain. Maybe I should consider using a neural network to answer those, it happens quite frequently that I need 2-3 rounds to get through reCaptcha.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#47

Post by plaidfuji »

joe_the_user wrote: post #21
dpandya wrote: post #11

It seems that the primary contribution of this technique is that it uses specific assumptions supported by neuroscience research in order to allow for composability of learning and better generalization. By introducing these specific assumptions (e.g. contours define objects), they are able to reduce the complexity that the model has to learn and thereby reduce the amount of data that it needs. Obviously, the questio…

After six or seven click-throughs, I downloaded the PDF. I haven't read it but skimming, I could see that there definitely were no formulas in it at all . Which sort of says, at best what it tells you is "we did this thing, which is kind of like X and kind of like Y with Z changes". Essentially, no way to reproduce or understand by itself. The first reference then had a link behind a paywall... So despite lots of app…

Haha yeah Science papers are about providing a high level explanation of what you did, in real words. Then you hit em with the 100 page supplemental that's got more detail than 3 papers' worth of research in other journals

Re: RCN is much more data efficient than traditional Deep Neural Networks

#48

Post by amelius »

dx034 wrote: post #45
Filligree wrote: post #44

Earlier quoted context omitted.

Those are letters?

I had no issue recognising any of those immediately.

Interestingly, the article says:

> Neuroscience evidence indicates that contours and surfaces are represented in a factored manner in the brain [8-11], which might be why people have no difficulty imagining a chair made of ice.

Re: RCN is much more data efficient than traditional Deep Neural Networks

#49

Post by sdenton4 »

BucketSort wrote: post #4

I'd love to read this, but the faint text on white background... good god. I went through the code looking to change the background so I could read it and found this: body { text-rendering: optimizeLegibility; } Ok

You're a datapoint in the upcoming Science paper, "RCN is Much Better at Reading Our Announcements Concerning RCN Than Actual Humans."

Re: RCN is much more data efficient than traditional Deep Neural Networks

#50

Post by creo »

BucketSort wrote: post #4

I'd love to read this, but the faint text on white background... good god. I went through the code looking to change the background so I could read it and found this: body { text-rendering: optimizeLegibility; } Ok

I wasn't sure why i cant focus on article until i've read this.
Post reply on HN