Is there an equivalent case of supernormal stimuli for NLP?
Fooling Neural Networks [pdf]
21–30 of 39 posts
Re: Fooling Neural Networks [pdf]
#22Is there an equivalent case of supernormal stimuli for NLP?
Natural language is a bit too complex for that I think.
There can't be any universal stimuli simply because there's multiple languages and cultures that don't all have the same response to stimuli.
There's a relationship between neural activity and writing system, for example [0].
Then there's stimuli that activate the language centre in some languages (e.g. click-sounds the Khoisan language families in Africa) but not in others. Some languages (especially East Asian languages like Vietnamese) also use tone to distinguish lexical or grammatical meaning, while Indo-European languages do not. which is another significant difference in (here: verbal) language processing.
All this leads me to conjecture that supernormal stimuli are highly unlikely in this context due to the high-level nature of the subject as well as the differences and the diversity in the involved regions of the brain.
[0] https://www.sciencedirect.com/science/article/abs/pii/S01680...
Re: Fooling Neural Networks [pdf]
#23I have a background in classic image processing and machine vision and back in the olden days we had the opposite problem: algorithms were just too specific to build useful applications. It's easy to detect lines and circles with a Hough Transform or do template matching for features that very closely match a sample. However, working up the chain it never came together, detecting cars in a parking lot, a relatively s…
A library to do this that integrated well with typical inference and and training systems would be a great contribution!
Re: Fooling Neural Networks [pdf]
#24Earlier quoted context omitted.
I think machine learning researchers are well aware that successful optimisation is only possible using the right priors. This is explicit in bayesian machine learning but also implicit in neural networks in the choice of the architecture, optimisation algorithm and hyper parameters. It's a well discussed problem and a lot of researchers have a serious background in optimisation, theoretical machine learning and othe…
What exactly are the right priors for general intelligence? And keep in mind, whichever prior you choose, I can design learning problem where it will lead you astray. This paper provides some interesting results on the weakness inherent in universal priors: https://arxiv.org/abs/1510.04931
That's for reinforcement learning, right? What is the adversarial learning problem in say, classification based on Solomonoff?
If hypercomputation is possible, then anything based on Kolmogorov complexity would be SOL, but if not... is Solomonoff induction just too expensive in practice?
Re: Fooling Neural Networks [pdf]
#25Earlier quoted context omitted.
What exactly are the right priors for general intelligence? And keep in mind, whichever prior you choose, I can design learning problem where it will lead you astray. This paper provides some interesting results on the weakness inherent in universal priors: https://arxiv.org/abs/1510.04931
Related question: What are the adversarial examples for human intelligence? We know some for the visual and auditory systems, but what about the arguably general intelligence of humans? Maybe we can work our way backwards from the adversarial examples to the inductive biases?
I think fractional reserve banking has done a pretty good job of fooling everyone.
Re: Fooling Neural Networks [pdf]
#26Earlier quoted context omitted.
What exactly are the right priors for general intelligence? And keep in mind, whichever prior you choose, I can design learning problem where it will lead you astray. This paper provides some interesting results on the weakness inherent in universal priors: https://arxiv.org/abs/1510.04931
Related question: What are the adversarial examples for human intelligence? We know some for the visual and auditory systems, but what about the arguably general intelligence of humans? Maybe we can work our way backwards from the adversarial examples to the inductive biases?
'Thinking Fast and Slow' is basically all about the rough edges of human thinking.
The interesting tradeoff with ML systems is that you trade lots of individual human crap for one big pile of machine crap. The advantage of the machine crap is that you can actually go in and find systemic problems and work on fixing them at a 'global' level. On the human side, you're always going to be stuck with an unknown array of individual human biases which are incredibly difficult to correct.
Re: Fooling Neural Networks [pdf]
#27Looking from far far away at the examples it seems networks are still trained towards a local optimum and are easily tripped. Basically the equivalent of a card trick failing horribly the moment audience enters the stage.
Re: Fooling Neural Networks [pdf]
#28Can someone please ELI5 why we can't just blur every image before passing it to the classifier? Wouldn't this defeat this sort of attack? Obviously you lose some accuracy, but that seems acceptable. Edit: I guess that's similar to "image quilting" (whatever that is) in this slide deck. This is the first time I've seen something like this mentioned. Seems like a straight-forward solution.
It turns out that a similar technique, where you basically apply noise multiple times to a single image, and average predictions over all noisy images- equivalent to convolving your nn with Gaussian noise yields near state of the art bounds on provable robustness (under a specific class of attacks). The issue is the magnitude of noise you need in order to get practically robust networks is quite large relative to the…
These are the papers I've been wondering why didn't exist ;)
Thanks!
Re: Fooling Neural Networks [pdf]
#29I have a background in classic image processing and machine vision and back in the olden days we had the opposite problem: algorithms were just too specific to build useful applications. It's easy to detect lines and circles with a Hough Transform or do template matching for features that very closely match a sample. However, working up the chain it never came together, detecting cars in a parking lot, a relatively s…
Because the new wave don't know how to do this. Deep Learning as AI was sold as "statistics in place of physics" -- ie., no need to understand reality (the target domain), simply brute force it.
The peddlers of such a message were bamboozled by early successes and lacked sufficient experience of empirical science to realise this was never going to work.
No NN will discover the universal law of gravitation from any dataset not collected on the basis of knowing this universal law. With 'statistics as theory' there can never be new theory, as a new theory is a precondition of a new dataset.
Re: Fooling Neural Networks [pdf]
#30Fooling darknet
https://imgur.com/a/EXHEIH5