this is actually a really good explanation because it shows in simple pictures and words the concepts so there is no confusion. Well done.
Random Forests for Complete Beginners
11–20 of 39 posts
Re: Random Forests for Complete Beginners
#12While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.
Re: Random Forests for Complete Beginners
#13While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.
On the other hand, if it works it works.
There are also lots of good ways to peer into the inner workings of a tree ensemble model nowadays. It's not laid out plainly for you like a linear model, but it's not an impenetrable black box as people like to suggest.
Re: Random Forests for Complete Beginners
#14Thanks for posting this - from a fellow HH member who saw this yesterday!
Re: Random Forests for Complete Beginners
#15My thesis is related to trees, forests, and ensemble of forests. This is pretty concise but mostly for decision tree but only half of it. CART is the framework for decision tree for classification and regression. This article only addresses the Classification part which usually use Gini which is a class of split that split along parallel axises (there are oblique trees). The regression part uses more traditional stat…
Hey! Author here. Appreciate the feedback.
You sound like you know what you're talking about. My site is open source, and if you want you can make a few edits to this article and submit a pull request! https://github.com/vzhou842/victorzhou.com/blob/master/conte...
Re: Random Forests for Complete Beginners
#16Hey, Author here. If you're new to ML you might also like my introduction to Neural Networks: https://victorzhou.com/blog/intro-to-neural-networks/
Discussion of my neural networks post on HN: https://news.ycombinator.com/item?id=19320217
Re: Random Forests for Complete Beginners
#17My thesis is related to trees, forests, and ensemble of forests. This is pretty concise but mostly for decision tree but only half of it. CART is the framework for decision tree for classification and regression. This article only addresses the Classification part which usually use Gini which is a class of split that split along parallel axises (there are oblique trees). The regression part uses more traditional stat…
is there a better tutorial/course for a beginner into this field ? the end goal not being academia, but being able to think and write reasonable production code.
> is there a better tutorial/course for a beginner into this field ?
I have no clue. I have a degree in Computer Science and am finishing my master in Applied Statistic. It just happen my skill set, statistical modeling, have many overlap with Machine Learning.
General overview maybe https://r4ds.had.co.nz/
But for straight up modeling I would start and recommend this book: https://otexts.org/fpp2/ (update: this book doesn't go over EDA, outlier, and such... r4ds does EDA. For outlier and imputation some statistic book can cover that.)
It's for time series modeling (statistical slanted) but I think it goes over modeling aspect really well and it very intro (coursera have a step above that book for a little more depth in time series). To be fair... statistical modeling for univariate time series statistical model is number one-ish see m4 competition or the uber blog on time series.
> the end goal not being academia, but being able to think and write reasonable production code.
For the thinking part which affect writing code, from my experiences (I can be wrong), data science/ML approach to modeling is different than statistic.
The vast majority of the time Data Science/ML are given the data which is why I believe AI algorithm can be bias (see Gyfcat and Asian computer vision classification problem). Where as statistic you usually figure out your hypothesis or what you are trying to answer and then you create a design experiment or strategy on how to collect the data without being bias and hopefully controlling factors. But statistic also do given data but the vast majority of models out there is slanted toward explanatory vs forecasting/prediction. DS/ML seems to care more about forecasting/prediction.
I also think how each discipline approach modeling affect the way a person think within that field too. I have not figure out the unified thinking of how ML/DS discipline approach model but I am confidence that it's not the same as statistic. But speaking from statistic vs applied math, I can give an example. For time series data, statistician model on the assumption that all we're given is the data and we'll try to extract every bit of information out of the data and explain away the variance with models (each predictor can remove away the noise by explaining it whatever left is error/chances). It's more data focus. For applied math, they figure out how the data is generated eg their model assume this is how the data is generated so they got these stochastic processes and is uses more toward probability than statistic.
So thinking would affect code... So how statistician code stuff is probably different than ML/DS. I've seen people calling imputation witchcraft and throw temporal data in random forest >___>.
> write reasonable production code.
I do R for modeling.
If you're not creating any new fancy algorithm, you can just use packages. You train them on the data set, and just ship the trained model. You can wrap it as a REST service via https://www.rplumber.io/. I like to think Python have something similar?
Do note I know very little about deep learning or how to ship that. I've chosen to specialize in statistical modeling and R for modeling.
Re: Random Forests for Complete Beginners
#18While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.
On the other hand, if it works it works. There are also lots of good ways to peer into the inner workings of a tree ensemble model nowadays. It's not laid out plainly for you like a linear model, but it's not an impenetrable black box as people like to suggest.
That’s true, but they also tend not to know them. Or worse, they point to variable importance and don’t know how it works which leads them to the wrong conclusion.
Re: Random Forests for Complete Beginners
#19While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.
The real differentiation will come from feature engineering. Random forests and boosted trees (the other fire-and-forget model of choice) can tell you a lot about the data set from their validation performance under tuning. It might be that you've got a straightforward situation, but often the next step is to dig into what the model is doing and see if you can better optimize the features.
Re: Random Forests for Complete Beginners
#20While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.
I am probably one of the junior DS you are referring to. But, I genuinely want to know the reason of using anything other than gradient boost tree to do classification on structured data.