Keras reimplementation of "One pixel attack for fooling deep neural networks"
1–10 of 84 posts
Re: Keras reimplementation of "One pixel attack for fooling deep neural networks"
#2Aren't these attacks just proof of the fact that even deep neural networks are approximating a high dimensional function by clamping the entropy of the formula, rather than truncating the range of input/output values?
For a 32x32 image, the space of 1-pixel attacks is 0xFFFFFF * 32 * 32 = 17179868160 = e^23
Expecting an input space as large as that to not poke through the entropically deprived network is destined to fail.
Re: Keras reimplementation of "One pixel attack for fooling deep neural networks"
#3Aren't these attacks just proof of the fact that even deep neural networks are approximating a high dimensional function by clamping the entropy of the formula, rather than truncating the range of input/output values? For a 32x32 image, the space of 1-pixel attacks is 0xFFFFFF * 32 * 32 = 17179868160 = e^23 Expecting an input space as large as that to not poke through the entropically deprived network is destined to…
I mean, you don't even need a proof of that. The latter is impossible since the range of input/output values is untruncated by construction?
Re: Keras reimplementation of "One pixel attack for fooling deep neural networks"
#4Aren't these attacks just proof of the fact that even deep neural networks are approximating a high dimensional function by clamping the entropy of the formula, rather than truncating the range of input/output values? For a 32x32 image, the space of 1-pixel attacks is 0xFFFFFF * 32 * 32 = 17179868160 = e^23 Expecting an input space as large as that to not poke through the entropically deprived network is destined to…
Brick shattering through window hypothesis finally has evidence
No seriously - am I missing something?
"Recent research has revealed that the output of Deep Neural Networks (DNN) can be easily altered by adding relatively small perturbations to the input vector."
"Submitted on 24 Oct 2017 "
Re: Keras reimplementation of "One pixel attack for fooling deep neural networks"
#5How well does this fair over several frames?
Re: Keras reimplementation of "One pixel attack for fooling deep neural networks"
#6Can we use one pixel attack to train network? We would generate adversarial examples to train network.
Having ability to create Training Set that maximizes learning factor for NN sounds amazing but I think we would run to other adversarial examples.
Re: Keras reimplementation of "One pixel attack for fooling deep neural networks"
#7Can we use one pixel attack to train network? We would generate adversarial examples to train network. Having ability to create Training Set that maximizes learning factor for NN sounds amazing but I think we would run to other adversarial examples.
Augmenting your training dataset with adversarial examples is known as adversarial training, see e.g. [0] for a recent overview with empirical results. This seems to be a good first step in defending against such attacks, though the most naive approach of adversarial training doesn't work as well as you'd expect.
Re: Keras reimplementation of "One pixel attack for fooling deep neural networks"
#8Aren't these attacks just proof of the fact that even deep neural networks are approximating a high dimensional function by clamping the entropy of the formula, rather than truncating the range of input/output values? For a 32x32 image, the space of 1-pixel attacks is 0xFFFFFF * 32 * 32 = 17179868160 = e^23 Expecting an input space as large as that to not poke through the entropically deprived network is destined to…
Brick shattering through window hypothesis finally has evidence No seriously - am I missing something? "Recent research has revealed that the output of Deep Neural Networks (DNN) can be easily altered by adding relatively small perturbations to the input vector." "Submitted on 24 Oct 2017 "
My guess here is that we need to learn more about adding noise to datasets before they go into a neural network. Lots of existing work do alterations like transposing, rotating, flipping and so on. Adding noise, including known attacks, would be part of this.
Re: Keras reimplementation of "One pixel attack for fooling deep neural networks"
#9Earlier quoted context omitted.
Brick shattering through window hypothesis finally has evidence No seriously - am I missing something? "Recent research has revealed that the output of Deep Neural Networks (DNN) can be easily altered by adding relatively small perturbations to the input vector." "Submitted on 24 Oct 2017 "
My guess here is that we need to learn more about adding noise to datasets before they go into a neural network. Lots of existing work do alterations like transposing, rotating, flipping and so on. Adding noise, including known attacks, would be part of this.
Noise will definitely help if random noise is added to every input image before the neural network sees it, in production. Essentially, a pre-input pipeline state that a bad actor cannot predict, making single pixel deception totally neutered.
I think what needs to be done here is to add a threshold of correlation between the input pixels. Consider that the problem is that 1 pixel change in the deviously right way, can be equivalent to the change when multiple pixels are changed in the proper way -- the derivative of the cost function. So clearly there needs to be a way to design / tell the network that 1 pixel change cannot be nearly as strong as changing multiple pixels relationships, in terms of cost function value change.
From what I can garner, the only way to accomplish this is to make sure the number of nodes in the hidden layers is strictly monotonically decreasing. By using the last layer as a "grab bag" for classification, with 100s of nodes greater than the previous layers, the network becomes vulnerable to single pixel attacks. There have to be ways to design classification styles networks without the fan-out.
Re: Keras reimplementation of "One pixel attack for fooling deep neural networks"
#10Aren't these attacks just proof of the fact that even deep neural networks are approximating a high dimensional function by clamping the entropy of the formula, rather than truncating the range of input/output values? For a 32x32 image, the space of 1-pixel attacks is 0xFFFFFF * 32 * 32 = 17179868160 = e^23 Expecting an input space as large as that to not poke through the entropically deprived network is destined to…
I mean, you don't even need a proof of that. The latter is impossible since the range of input/output values is untruncated by construction?
I'm very new to ML, so I understand about 50% of what @eximius and @goldenkey are saying, but definetly not 100%. can anyone explain it in a bit more detail? (im assuming "entropy" is the key concept i need to put on my learning queue.)
“approximating a high dimensional function by clamping the entropy of the formula, rather than truncating the range of input/output values”
“not poke through the entropically deprived network is destined to fail”
“the range of input/output values is untruncated by construction”