Fooling Neural Networks [pdf]
slazebni.cs.illinois.edu
Fooling Neural Networks [pdf]
1–10 of 39 posts
Re: Fooling Neural Networks [pdf]
#2The input space for these neural networks is huge, it is roughly the number of colors to the power of the number of pixels. What neural networks do is subdivide the input space and assign a label to it. Because of the high dimension of the input space it is very likely that it is possible to find images that are on the boundary between two labels. Using more advanced techniques might make it more difficult for an adversary to find such examples, but it does not eliminate their existence.
One of the big problems with neural networks (and other AI techniques as well) is that they cannot explain their classifications, which makes it difficult to determine whether a classification is correct. Most people seriously underestimate how difficult this task is. Humans can do it quite easily because our hardware has been optimized by eons of evolution. Neural networks are only in their infancy.
Re: Fooling Neural Networks [pdf]
#3The input space for these neural networks is huge, it is roughly the number of colors to the power of the number of pixels. What neural networks do is subdivide the input space and assign a label to it. Because of the high dimension of the input space it is very likely that it is possible to find images that are on the boundary between two labels. Using more advanced techniques might make it more difficult for an adv…
The problem goes much deeper than these adversarial examples. The main issue is Solomonoff Uncomputability (or the No Free Lunch in Search and Optimization theorem, or any of the other hard limiting theorems).
In short, it’s not only that you can devise adversarial examples that find the blindspots of the function approximator and fool it into misprediction, it’s that for any learning optimization algorithm you can abuse its priors and biases and create an environment in which it will perform terribly. This is a fundamental and inherent feature of how we go about machine learning — equating it with optimizing functions — and we will need a paradigm shift to go around it.
It’s curious to me how most of these results are known for decades, yet most researchers seem dead set on ignoring them.
Re: Fooling Neural Networks [pdf]
#4Newer slides from Spring 2021 class - https://slazebni.cs.illinois.edu/spring21/lec13_adversarial....
Re: Fooling Neural Networks [pdf]
#5The input space for these neural networks is huge, it is roughly the number of colors to the power of the number of pixels. What neural networks do is subdivide the input space and assign a label to it. Because of the high dimension of the input space it is very likely that it is possible to find images that are on the boundary between two labels. Using more advanced techniques might make it more difficult for an adv…
The problem goes much deeper than these adversarial examples. The main issue is Solomonoff Uncomputability (or the No Free Lunch in Search and Optimization theorem, or any of the other hard limiting theorems). In short, it’s not only that you can devise adversarial examples that find the blindspots of the function approximator and fool it into misprediction, it’s that for any learning optimization algorithm you can a…
I think machine learning researchers are well aware that successful optimisation is only possible using the right priors. This is explicit in bayesian machine learning but also implicit in neural networks in the choice of the architecture, optimisation algorithm and hyper parameters. It's a well discussed problem and a lot of researchers have a serious background in optimisation, theoretical machine learning and other related areas.
Re: Fooling Neural Networks [pdf]
#6Can’t we use the same method to generate adversarial inputs to iteratively train multiple model? After each model is generated we expand the data set by using the prior model to generate the adversarial inputs and then train a classifier maximizes the performance on both the inputs and adversarial inputs.
Now we just use n models in production and use voting for produce the label.
As n gets large, does this become robust to adversarial inputs?
Re: Fooling Neural Networks [pdf]
#7Can’t we use the same method to generate adversarial inputs to iteratively train multiple model? After each model is generated we expand the data set by using the prior model to generate the adversarial inputs and then train a classifier maximizes the performance on both the inputs and adversarial inputs. Now we just use n models in production and use voting for produce the label. As n gets large, does this become ro…
Teacher student!!
Re: Fooling Neural Networks [pdf]
#8Anyone have a sense of how much of a problem this is?
It's not surprising that a network can be fooled by small input changes, but if some image preprocessing is enough to solve this It's not a big problem.
On the other hand, if I can make a sign that looks like a stop sign to people but looks like a road work sign to a tesla, that's obviously a big deal.
These slides touch on the difference by saying that physical examples of adversarial inputs are harder, and they mention some mitigation techniques, but they don't seem to really quantify how effective mitigation is in real world scenarios.
Re: Fooling Neural Networks [pdf]
#9I have a background in classic image processing and machine vision and back in the olden days we had the opposite problem: algorithms were just too specific to build useful applications. It's easy to detect lines and circles with a Hough Transform or do template matching for features that very closely match a sample. However, working up the chain it never came together, detecting cars in a parking lot, a relatively simple task with CNNs, was a very difficult problem.
I wonder if enough work is being done to combine the achievements of each field. Whenever I see adversarial examples I wonder why people aren't doing more preprocessing to root out obvious problems with normalization in scale, color, perspective, etc. Also, if we could feed networks with higher level descriptors instead of feeding low-information-density color images, wouldn't that make life easier.
I'm sure I'm not the only one thinking this, is there any good research being done in that space?
Re: Fooling Neural Networks [pdf]
#10Earlier quoted context omitted.
The problem goes much deeper than these adversarial examples. The main issue is Solomonoff Uncomputability (or the No Free Lunch in Search and Optimization theorem, or any of the other hard limiting theorems). In short, it’s not only that you can devise adversarial examples that find the blindspots of the function approximator and fool it into misprediction, it’s that for any learning optimization algorithm you can a…
I think machine learning researchers are well aware that successful optimisation is only possible using the right priors. This is explicit in bayesian machine learning but also implicit in neural networks in the choice of the architecture, optimisation algorithm and hyper parameters. It's a well discussed problem and a lot of researchers have a serious background in optimisation, theoretical machine learning and othe…
What exactly are the right priors for general intelligence? And keep in mind, whichever prior you choose, I can design learning problem where it will lead you astray.
This paper provides some interesting results on the weakness inherent in universal priors: https://arxiv.org/abs/1510.04931