61–67 of 67 posts
This kind of stuff is fairly normal actually. I don't really agree that it couldn't happen. Neural Networks train for the wrong features all of the time. It's part of what happens when you're training unsupervised. You load in a lot of data and then you find the bias. Sure, as the article purports, if done perfectly it wouldn't happen. But that's like saying "If you build a bridge perfectly it won't fall down." befor…
>I suggest that dataset bias is real but exaggerated by the tank story, giving a misleading indication of risks from deep learning I don't see how this story gives a "misleading" view of deep learning. From my (admittedly limited) experience with self-driving RC cars, this type of mistake is quite easy for a neural net to make while being quite difficult to detect. In our case, after utilizing a visual back-prop meth…
Another more modern and well-documented example of this would seem to occur in a 2015 write-up of the "Right Whale" competition in Kaggle: http://felixlaumon.github.io/2015/01/08/kaggle-right-whale.h... Contrary to this author's claims, despite using data augmentation and a fancy modern CNN, a neural network trained to identify whales hit a local optimum where it looked at patterns in waves on the water to identify t…
Me neither. Especially considering that this story was already alive before the latest deep learning advances. It is totally believable.
And even with a modern CNN approach, you would expect a model to be able to learn a sunny/cloudy categorization much easier than the nationality of a tank.
This story was repeated by professionals for ages because it is totally believable.
Earlier quoted context omitted.
But ensuring correct datasets is a problem that people new to machine learning have to be made aware of. It's easy for people to see that NNs are good with 'noisy' data and incorrectly assume they can throw any data at it and get good results. And I think the quote is a false dilemma -- its not like we can't have multiple different stories for different problems. Make up / find a "truthy" story to spread for misoptim…
If a failure mode has never been reported in the wild, why is it so important to tell a juicy story about it, at the expense of attention to empirically observed failure modes?
However, also consider that the tank parable has circulated in textbooks and undergraduate introductory lectures for several decades. The main lesson I learned from the story that after training a model, one should validate it to see if it does generalize to detecting tanks both during night and daytime. Is it really a surprise that it might be difficult find egregiously naive mistakes?
[1] http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.ht...
Earlier quoted context omitted.
That's why testing and validation sets exists. Overfitting is prevented by ending learning process when error on validation set starts growing. It is a standard procedure, so it's unlikely there's a recent case of overfitting slipping into production. If irrelevant feature is present in all class samples, then it is not a fault of NN to use it as a class feature, it's bad data.
So you would agree overfitting is not a real issue and talking about it is distracting from NN? My question is are people using overfitting as an excuse of a what is instead a badly made NN. If you are smart enough to create a NN that can tell if it's sunny or not then tanks would also be possible. But if your NN just sucks than blaming overfitting is a convenient out.
Overfitting is a major issue in machine learning and it's an inherent characteristic of learning from examples and not the result of a mistake, or of poor practice. There are special techniques developed explicitly to reduce overfitting- early stopping (what red75prime above, describes), regularisation, bagging (in decision trees) etc. A lot of work also goes into ensuring measures of learning performance don't mistake overfitting for successful learning (e.g. k-fold cross validation).
I'm sorry that I don't have time to track down a good source for a discussion of the bias-variance tradeoff and overfitting. You can start at the wikipedia page [https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff] and follow the links. In short- a model that learns to reproduce its example data with very high fidelity, risks generalising poorly, whereas a model that generalises well may have high training error. Linear classifiers in particular are high-bias, whereas nonlinear learners, like multi-layered neural networks or decision trees, are high-variance.
The problem is real, it's a big bugbear and you won't find any specialist who dismisses it, or who considers it "not a real issue".
Earlier quoted context omitted.
That's why testing and validation sets exists. Overfitting is prevented by ending learning process when error on validation set starts growing. It is a standard procedure, so it's unlikely there's a recent case of overfitting slipping into production. If irrelevant feature is present in all class samples, then it is not a fault of NN to use it as a class feature, it's bad data.
So you would agree overfitting is not a real issue and talking about it is distracting from NN? My question is are people using overfitting as an excuse of a what is instead a badly made NN. If you are smart enough to create a NN that can tell if it's sunny or not then tanks would also be possible. But if your NN just sucks than blaming overfitting is a convenient out.
Over-fitting is dismissed as an amateur mistake when it really is an endemic problem you are constantly battling no matter how good you are.
Earlier quoted context omitted.
That's an interesting variant - none of the versions I've seen so far link it to Minsky's perceptron book. Any chance you recall where you saw that one?
This was from a prof giving an undergrad ML lecture at Cornell about 10-12 years ago. Wikipedia's coverage of _Perceptrons_ suggests my lecturer also had only heard the mistaken version of Minky's XOR example, so this could have been entirely wrong :)
The whole "Could it happen"? section is a bit strange. On the one hand, it focuses on CNNs when it's clear we're talking about a binary classifier (the article itself points that out). If Fredkin was really the originator of the story, then discussing CNNs is an anachronism (they were 30 years away at the time). More importantly, it's obvious that "it" could definitely happen and in fact happens a lot- "it" being ove…
? How is it strange? Presumably people are not, right now, still retelling the story because they are terribly afraid that there are perceptrons out there from the 1960s lurking, waiting to strike, or that anyone is going to go out and try to use 1960s style perceptrons. People are telling it as a cautionary story about current NNs, in the 2000s and 2010s and 2017. Which means... CNNs. So it's worth asking, can it happen with CNNs as trained by any reasonably standard workflow?
> More importantly, it's obvious that "it" could definitely happen and in fact happens a lot- "it" being overfitting to examples.
Overfitting is not dataset bias, as I note several times. For example, dropout or heldout datasets or crossvalidation are highly effective in fighting/detecting overfitting, but do nothing about dataset bias.