aside: There really needs to be a law :) capping cpu cycles / information gained. Energy ain't free. Emissions are not harmless. OP appears to be a gas-guzzler algorithm.
The Unreasonable Effectiveness of Random Forests
11–20 of 35 posts
Re: The Unreasonable Effectiveness of Random Forests
#12This is so true. Random forest should be the default choice for most problem sets. Now we have The Unreasonable Effectiveness of Random Forests and The Unreasonable Effectiveness of Recurrent Neural Networks . We just need The Unreasonable Effectiveness of XGBoost (for winning Kaggle competitions) and we'll have the whole set. [1] http://karpathy.github.io/2015/05/21/rnn-effectiveness/
Maybe we need a RNN analyzing titles of classics to come up with other tentative titles (forgot how this process is called)
Re: The Unreasonable Effectiveness of Random Forests
#13I wish there were some examples in the article of their effectiveness, mentions of best areas for their use, etc. Currently, as a non-ML expert, my takeaway from the article was "there's a thing called a Random Forest. That's quite cool." Maybe I'm not the target audience, though.
Random Forest is an algorithm for classification or prediction.
Usually the SciKit-learn algorithm cheat sheet[1] is a good guide for this, but it doesn't include random forest (or any kind of decision tree).
[1] http://scikit-learn.org/stable/tutorial/machine_learning_map...
Re: The Unreasonable Effectiveness of Random Forests
#14One of thing to watch out for when using RFs is when your test set doesn't have identical factor levels as your training data. Often can actually lead to a significant bias in the prediction. Can't find a great paper on this but some discussion here:http://stats.stackexchange.com/questions/29446/random-forest...
Re: The Unreasonable Effectiveness of Random Forests
#15A bigger-picture "unreasonable effectiveness" article would be "The Unreasonable Effectiveness of Weighted Majority Votes in Machine Learning."
This Monday I'm publishing a post on [1] that gives some rigorous explanation for this, which essentially covers the results of this paper [2] whose main theorem is a claim about majority voting schemes.
[2]: http://cseweb.ucsd.edu/~yfreund/papers/BoostingtheMargin.pdf
Re: The Unreasonable Effectiveness of Random Forests
#16I'm surprised the author didn't mention model variance reduction, which is what RFs were designed to do. For example, if we generate 100 data sets which are the same except for noise, the 100 decision boundaries drawn by a classification tree will vary much more than the decision boundaries drawn by RFs on the same sets.
Here's what this means visually:
http://i.imgur.com/IjfXFkm.png
There is just one of the 100 data sets generated shown.
Re: The Unreasonable Effectiveness of Random Forests
#17aside: There really needs to be a law :) capping cpu cycles / information gained. Energy ain't free. Emissions are not harmless. OP appears to be a gas-guzzler algorithm.
What are you even talking about ??
Re: The Unreasonable Effectiveness of Random Forests
#18This is so true. Random forest should be the default choice for most problem sets. Now we have The Unreasonable Effectiveness of Random Forests and The Unreasonable Effectiveness of Recurrent Neural Networks . We just need The Unreasonable Effectiveness of XGBoost (for winning Kaggle competitions) and we'll have the whole set. [1] http://karpathy.github.io/2015/05/21/rnn-effectiveness/
Another one is The Unreasonable Effectiveness of Deep Learning [1] by LeCun. [1] http://on-demand.gputechconf.com/gtc/2014/webinar/gtc-expres...
All of which are a play on 'The Unreasonable Effectiveness of Mathematics in the Natural Science,' I believe.
https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness...
Re: The Unreasonable Effectiveness of Random Forests
#19I share the awe of RF (especially as they look naive, but turn out to be extremely good for a wide range of problems), however, the main problem is that they are black boxes. Typically, it is easy to get good results, but almost no insight, or further pointers. Many times I ended up using linear regression (with properly engineered variables), or something as simple, because it gave almost as good results as RF, but…
This is pretty much exactly the message of a talk I attended at CU Boulder. The Microsoft researcher spoke of how linear regression for the classification of whether a given medical patient will be re-admitted to the hospital offered much more valuable insight into the relationships different diseases had to the probability of re-admittance.
Moreover, not-so-obviously strange rules learned by more complex models were exposed as strange by the information-dense pictorial representation of linear regression.
Re: The Unreasonable Effectiveness of Random Forests
#20A bigger-picture "unreasonable effectiveness" article would be "The Unreasonable Effectiveness of Weighted Majority Votes in Machine Learning." This Monday I'm publishing a post on [1] that gives some rigorous explanation for this, which essentially covers the results of this paper [2] whose main theorem is a claim about majority voting schemes. [1]: http://jeremykun.com/ [2]: http://cseweb.ucsd.edu/~yfreund/papers/B…
Thanks, [2] is a very nice paper with intuitive ideas and insights. To summarize it: When the common people agree and few disagree there are great expectations that a generalization is possible and effective.