Benchmarking Random Forest Classification
about.wise.io
Benchmarking Random Forest Classification
1–8 of 8 posts
Re: Benchmarking Random Forest Classification
#2Its random forests ... each tree is trained on a subset of the data. You can split the massive dataset into chunks and train independently. That sidesteps the "big data" hangup.
If you look at the implementation for ski-learn, each tree emits a normalised probability vector for each prediction, those vectors are simply multiplied together to get the aggregate prediction, so its not very difficult to do yourself.
Although regardless, you are applying a batch learning technique anyway. You want an incremental learner for big data.
Re: Benchmarking Random Forest Classification
#3Question: Why do I have to implement hyperparameter selection?
For me, the promise of in-the-cloud machine learning is that I can call 'train' method, and specify one single hyperparameter: training budget (i.e. $). Perhaps also the max time before I am returned a trained model.
That's it. Can you do that?
Re: Benchmarking Random Forest Classification
#4Question: Why do I have to implement hyperparameter selection? For me, the promise of in-the-cloud machine learning is that I can call 'train' method, and specify one single hyperparameter: training budget (i.e. $). Perhaps also the max time before I am returned a trained model. That's it. Can you do that?
There are different ways of finding optimal hyperparameters, and while a cloud system might be capable to provide a mechanism that can figure this out on its own, generally this will be less efficient...
Re: Benchmarking Random Forest Classification
#5Its random forests ... each tree is trained on a subset of the data. You can split the massive dataset into chunks and train independently. That sidesteps the "big data" hangup. If you look at the implementation for ski-learn, each tree emits a normalised probability vector for each prediction, those vectors are simply multiplied together to get the aggregate prediction, so its not very difficult to do yourself. Alth…
I was thinking the same thing. Ensemble methods like this scale very well with the number of machines, with moderate efforts of coordination at least.
Re: Benchmarking Random Forest Classification
#6Its random forests ... each tree is trained on a subset of the data. You can split the massive dataset into chunks and train independently. That sidesteps the "big data" hangup. If you look at the implementation for ski-learn, each tree emits a normalised probability vector for each prediction, those vectors are simply multiplied together to get the aggregate prediction, so its not very difficult to do yourself. Alth…
The training subset for each tree can still be quite large. Note that most of the implementations failed on their 12 GB dataset.
Although I'm a big believer in streaming/online machine learning, it's not necessarily the best solution. There are many cases when batch is the better option, especially for big data. Anything historical, really.
Re: Benchmarking Random Forest Classification
#7Question: Why do I have to implement hyperparameter selection? For me, the promise of in-the-cloud machine learning is that I can call 'train' method, and specify one single hyperparameter: training budget (i.e. $). Perhaps also the max time before I am returned a trained model. That's it. Can you do that?
This is exactly what we're enabling with our ML Platform (currently in private beta). Such a system needs to be built on top of fast & scalable ML technology with smart & efficient tuning/optimization.
Would love to hear about your use cases & get you on the beta.
-Joey Richards, Chief Scientist @ wise.io
Re: Benchmarking Random Forest Classification
#8Any chance for you to run your benchmarks on this branch of Scikit-Learn? https://github.com/glouppe/scikit-learn/tree/trees-v2 This will be shipped anytime soon :)
We have been working hard to reduce computing times and memory footprint (though, there is still a lot of improvement on that side).
(Unfortunately, I cannot run your benchmarks myself, because the compiled version of WiseRF requires a newer version of glibc than the one on my cluster, and crashes.)