Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Random Forests for Complete Beginners

victorzhou.com

Post reply on HN
Print view View original

21–30 of 39 posts

Re: Random Forests for Complete Beginners

#21

Post by zachwill »

nerdponx wrote: post #13
b_tterc_p wrote: post #12

While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.

On the other hand, if it works it works. There are also lots of good ways to peer into the inner workings of a tree ensemble model nowadays. It's not laid out plainly for you like a linear model, but it's not an impenetrable black box as people like to suggest.

I completely agree. Tools like shap are really useful for peering into tree-based models.

https://github.com/slundberg/shap

Re: Random Forests for Complete Beginners

#22

Post by savagedata »

sandGorgon wrote: post #10
anthony_doan wrote: post #7

My thesis is related to trees, forests, and ensemble of forests. This is pretty concise but mostly for decision tree but only half of it. CART is the framework for decision tree for classification and regression. This article only addresses the Classification part which usually use Gini which is a class of split that split along parallel axises (there are oblique trees). The regression part uses more traditional stat…

is there a better tutorial/course for a beginner into this field ? the end goal not being academia, but being able to think and write reasonable production code.

I wrote this regression tree tutorial a few years back that might be a good complement to the tutorial above since it covers regression instead of classification and goes on to talk about bagging vs random forest, out-of-bag samples, and tuning parameters: https://github.com/savagedata/regression-tree-tutorial I wrote it at the start of my career and haven't shared it beyond my study group, so I'm happy to hear feedback.

Re: Random Forests for Complete Beginners

#23

Post by onlyrealcuzzo »

RavlaAlvar wrote: post #20
b_tterc_p wrote: post #12

While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.

I am probably one of the junior DS you are referring to. But, I genuinely want to know the reason of using anything other than gradient boost tree to do classification on structured data.

Is there a place that tells you: If you have this type of data and want this kind of answer, here's the best algorithm (and why)??

Re: Random Forests for Complete Beginners

#24

Post by rrggrr »

vzhou842 wrote: post #16

Hey, Author here. If you're new to ML you might also like my introduction to Neural Networks: https://victorzhou.com/blog/intro-to-neural-networks/ Discussion of my neural networks post on HN: https://news.ycombinator.com/item?id=19320217

Great contributions. Great work and I ought not ask for more, but gosh if you could put in real world examples with code it would be great.

Re: Random Forests for Complete Beginners

#26

Post by claytonjy »

sandGorgon wrote: post #10
anthony_doan wrote: post #7

My thesis is related to trees, forests, and ensemble of forests. This is pretty concise but mostly for decision tree but only half of it. CART is the framework for decision tree for classification and regression. This article only addresses the Classification part which usually use Gini which is a class of split that split along parallel axises (there are oblique trees). The regression part uses more traditional stat…

is there a better tutorial/course for a beginner into this field ? the end goal not being academia, but being able to think and write reasonable production code.

For decision trees, I really like http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ and https://explained.ai/decision-tree-viz/index.html.

For Random Forests, I like this one: https://www.gormanalysis.com/blog/random-forest-from-top-to-..., which also has a link to a decision-tree post. That blog also has the best GBM explainer I've seen yet (Gradient Boosted Machines are the _other_ tree-ensembling method in common use, where the trees are _stacked_ instead of _bagged_)

Your goal should not be to know enough to write an RF implementation, but rather to have some intuition behind how it works, so you can better choose when to use it or not. The likelihood of it ever making sense for you to write and RF algorithm for production use is extremely unlikely; use the great code that already exists for most languages.

Re: Random Forests for Complete Beginners

#27

Post by claytonjy »

b_tterc_p wrote: post #12

While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.

What's the bad sign there? If juniors are reaching for RF as a first-pass over say, (logistic) regression, that seems like a _great_ sign; no other approach has such a high average performance-effort ratio. For non-huge, naturally tabular problems, it usually takes me 10x or more the effort to beat the very first RF I train. Developer time matters!

If they start going to GBM's or neural nets first...I'd call _that_ a bad sign (and it happens).

Re: Random Forests for Complete Beginners

#28

Post by b_tterc_p »

RavlaAlvar wrote: post #20
b_tterc_p wrote: post #12

While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.

I am probably one of the junior DS you are referring to. But, I genuinely want to know the reason of using anything other than gradient boost tree to do classification on structured data.

It depends on your goal, and the nature of the problem. If you need to explain something in simplest terms, maybe use regularized logistic regression. If you need to make sensitive decisions, maybe a tree would be best because you have a clear sense of the variance in your answers at each node.

There’s nothing wrong with random forest. It’s a perfectly good model. But when it is someone’s only tool, it implies they both don’t know much much about the toolkit, and also how that one particular tool works.

I rarely use anything but linear models, trees and forests fwiw.

Re: Random Forests for Complete Beginners

#29

Post by gvand »

vzhou842 wrote: post #16

Hey, Author here. If you're new to ML you might also like my introduction to Neural Networks: https://victorzhou.com/blog/intro-to-neural-networks/ Discussion of my neural networks post on HN: https://news.ycombinator.com/item?id=19320217

Please don't stop and post other articles :)

Re: Random Forests for Complete Beginners

#30

Post by donjigweed »

onlyrealcuzzo wrote: post #23
RavlaAlvar wrote: post #20

Earlier quoted context omitted.

I am probably one of the junior DS you are referring to. But, I genuinely want to know the reason of using anything other than gradient boost tree to do classification on structured data.

Is there a place that tells you: If you have this type of data and want this kind of answer, here's the best algorithm (and why)??

https://scikit-learn.org/stable/tutorial/machine_learning_ma...
Post reply on HN