31–40 of 55 posts
This actually kind of freaks me out. Might it be possible that there is a way to corrupt brains?
This actually kind of freaks me out. Might it be possible that there is a way to corrupt brains?
We are well aware of various optical illusions, but are there "illusions" that be applied against other brain structures, such as memory or beliefs?
http://en.wikipedia.org/wiki/Lost_in_the_mall_technique
http://en.wikipedia.org/wiki/Memory_conformity
Not sure what you mean by beliefs.
There was also an experiment on split-brain patients (the connection between the left and right hemispheres is severed) where they'd show a command like "WALK" to the patient's right hemisphere only. They'd get up and walk. But since language is often localized to the left hemisphere, if you talk to them you are talking to the left hemisphere only, which did not see the command. Instead of saying something like "I don't know", they would make up a plausible reason to get up like "I'm getting a drink".
Does that count? It exploits what we know about the visual system (half of the visual field goes to each hemisphere) and localization of a particular function (language) in a patient with a specific disability (their corpus collosum is severed, which is mostly asymptomatic) to produce a completely strange result (invention of a motive).
Let's not forget that the word "imperceptible" is a heavily laden term in this context. There are numerous modifications to the data that would be "imperceptible" to a machine learning system, but would completely confuse a human. For example if you were to divide the image into a grid, and shuffle the squares, many ML systems would be tolerant to this kind of modification because some training regimes do this anyway…
This problem was observed 20+ years ago with linear models used for protein structure prediction. For any given model of what described a properly folded protein, one could locate conformations of the same protein that were rated as folded even better than the correct conformation (I called them doppelgangers, but the name "decoy" is what caught on). The statistical naivete of the field led to all sorts of inadverten…
> If it's likely overfitting, attempt an expectation maximization-like fix to the problem. Train the network. Generate adversaries, Add them to the training set, train again and then lather rinse repeat until either the network can't be trained or the problem goes away. As I quoted in my other comment, the paper suggests doing exactly that.
"The above observations suggest that adversarial examples are somewhat universal and not just the results of overfitting to a particular model or to the specific selection of the training set. They also suggest that back-feeding adversarial examples to training might improve generalization of the resulting models."
20 years ago I did this for linear models for protein energetics (also known as knowledge-based potentials or force fields), adding the decoys then refitting the parameters ad nauseum. What I eventually arrived at was the invalidation of every single energy model and force field in use for protein energetics (yes I really reverse engineered just about everyone from Michael Levitt to George Rose to AMBER, CHARMM, and ECEPP). This was an unpublishable result according to my post-doc adviser at the time so it never got written up.
In retrospect, he was utterly wrong. So I am really curious what would happen here if this were attempted with these much more complex models.
Not an example from deep learning, but [1] also demonstrates that Bayesian systems also have similar problems with sensitivity to initial conditions that are quite similar. It is also rather striking that these DLNs seem to be tricked by what we would typically think of as noise. 1. http://arxiv.org/abs/1308.6306
Earlier quoted context omitted.
> If it's likely overfitting, attempt an expectation maximization-like fix to the problem. Train the network. Generate adversaries, Add them to the training set, train again and then lather rinse repeat until either the network can't be trained or the problem goes away. As I quoted in my other comment, the paper suggests doing exactly that.
I'm knowingly being a pedant and I apologize for that but they don't quite say that, rather they come awfully close to doing so. And I'm being a pedant because of all the low information sorts claiming this is a refutation of deep neural networks (it's not, well at least not yet). "The above observations suggest that adversarial examples are somewhat universal and not just the results of overfitting to a particular m…
That said, I'm a bit surprised you found what you did about AMBER (and other force fields), or rather, that you didn't publish. The cornell et al force field was later acknowledged to have serious problems with protein folding, but a number of improvements have been made since then.
Anyway, I would have happily published that result with you (I worked with Kollman, have worked with Baker and Pande, and desperately want to see the force fields improve using machine learning). There was a guy at BMS who was working on this back in the day ('99-2000) who was using ML and the AMBER folks trashed him because they believed the force field's transferrability from small molecules to proteins was valid (in many ways it was, but it got some key details wrong).
Ifd you think there is a straightforward machine learning for force field problem that can dramatically improve ab initio folding with distbelief and exacycle, let me know. It shouldn't be hard to figure out my email address if you look at the papers I cited and do some basic set operations :-)
Let's not forget that the word "imperceptible" is a heavily laden term in this context. There are numerous modifications to the data that would be "imperceptible" to a machine learning system, but would completely confuse a human. For example if you were to divide the image into a grid, and shuffle the squares, many ML systems would be tolerant to this kind of modification because some training regimes do this anyway…
I disagree with your opinion. What this says to me is that DNN is not how humans classifies images.
Let's not forget that the word "imperceptible" is a heavily laden term in this context. There are numerous modifications to the data that would be "imperceptible" to a machine learning system, but would completely confuse a human. For example if you were to divide the image into a grid, and shuffle the squares, many ML systems would be tolerant to this kind of modification because some training regimes do this anyway…
[0] http://cs.nyu.edu/~zaremba/docs/understanding.pdf [1] http://puu.sh/9B4eG/1e9f7eb56b.png
Earlier quoted context omitted.
Adversarial examples are tied to a specific algorithm, they didn't produce any universal adversarial examples from what I understand.
From the article: What’s more surprising is that the same perturbation can cause a different network, which was trained on a different training dataset, to misclassify the same image. It means that adversarial examples are somewhat universal.
