I have a question about neural networks. Say, you are training a NN to recognize handwritten characters 0 and 1, and you have 1000 training images for each character (so 2000 images in total). All images are bitmaps with 0 for black and 1 for white. Now, by accident, all the "0" training-images have an even number of black pixels, and all the "1" training-images have an odd number of black pixels. How do you know tha…
This is more or less a case of overfitting. The algorithm works on the training set but doesn't generalize well. Tweaking the algorithm by decreasing the number of nodes in the hidden layer and doing cross-validation usually can help with this.