I've always found it curious that Neural Networks get so much hype when xgboost (gradient boosted decision trees) is by far the most popular and accurate algorithm for most Kaggle competitions. While neural networks are better for image processing types of problems, there are a wide variety of machine learning problems where decision tree methods perform better and are much easier to implement.
Deep Forest: Towards an Alternative to Deep Neural Networks
41–50 of 100 posts
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#42Earlier quoted context omitted.
I couldn't disagree more. The defaults don't just work, and the architecture of the network could also be considered a hyper parameter in which case what would be a reasonable default for all the types of problems ANN are used for?
Are you using batch normalization? If you are, an issue I see all the time is folks not setting the EMA filter coef correctly. In keras, it defaults to something like 0.99 which in my mind makes no sense. I use something around 0.6 and life is good. You want to get an overall good measurement of the statistics and in my mind the frequency cutoff when coef=0.99 is just way too high for most application. You usually wa…
Interesting, totally new concept for me: Where can I read more about EMA filter coefficient in Keras? My Google-fu is failing.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#43I've always found it curious that Neural Networks get so much hype when xgboost (gradient boosted decision trees) is by far the most popular and accurate algorithm for most Kaggle competitions. While neural networks are better for image processing types of problems, there are a wide variety of machine learning problems where decision tree methods perform better and are much easier to implement.
[deleted]
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#44I've always found it curious that Neural Networks get so much hype when xgboost (gradient boosted decision trees) is by far the most popular and accurate algorithm for most Kaggle competitions. While neural networks are better for image processing types of problems, there are a wide variety of machine learning problems where decision tree methods perform better and are much easier to implement.
Any good links you recommend learning xgboost? I've never quite figured out how they work.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#45I've always found it curious that Neural Networks get so much hype when xgboost (gradient boosted decision trees) is by far the most popular and accurate algorithm for most Kaggle competitions. While neural networks are better for image processing types of problems, there are a wide variety of machine learning problems where decision tree methods perform better and are much easier to implement.
Any good links you recommend learning xgboost? I've never quite figured out how they work.
[deleted]
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#46I've always found it curious that Neural Networks get so much hype when xgboost (gradient boosted decision trees) is by far the most popular and accurate algorithm for most Kaggle competitions. While neural networks are better for image processing types of problems, there are a wide variety of machine learning problems where decision tree methods perform better and are much easier to implement.
There are lots of reasons, principally among them that Kaggle comps problems represent only one tiny fraction of ML problems.
On general purpose ML, with out much time rigging a optimal solution, RF/Xgboost will preform better. But in many problems, ie vision, DL is vastly superior.
The other important point is that there many new opportunities for researchers regarding DL, where statistical ML approach on supervised problems is a much more well established field.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#47I've always found it curious that Neural Networks get so much hype when xgboost (gradient boosted decision trees) is by far the most popular and accurate algorithm for most Kaggle competitions. While neural networks are better for image processing types of problems, there are a wide variety of machine learning problems where decision tree methods perform better and are much easier to implement.
I think the hype around CNN is the NLP aspect of it as it relates to AI. If you can hammer down NLP and translate voice or text to computer-legible commands, you've really improved the user experience.
On the other side of the CNN coin is the image recognition that's getting a lot of hype from the self driving auto crowd. I think any data scientist worth their salt understands how the different algorithms stack up against each other. You wouldn't use xgboost for a computer vision problem just like you wouldn't use CNN for a tabular data problem.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#48I've always found it curious that Neural Networks get so much hype when xgboost (gradient boosted decision trees) is by far the most popular and accurate algorithm for most Kaggle competitions. While neural networks are better for image processing types of problems, there are a wide variety of machine learning problems where decision tree methods perform better and are much easier to implement.
There are lots of reasons, principally among them that Kaggle comps problems represent only one tiny fraction of ML problems. On general purpose ML, with out much time rigging a optimal solution, RF/Xgboost will preform better. But in many problems, ie vision, DL is vastly superior. The other important point is that there many new opportunities for researchers regarding DL, where statistical ML approach on supervised…
Also note that current "leader board" standard on MNIST is 99.8, not 99% he compares against.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#49The method's performance on MNIST is relatively mediocre. You might think 98.96% is amazing, but it's about relative performance. It is a relatively easy exercise nowadays to get above 99% with neural nets. Even I can get that kind of performance with hand-written Python neural nets, on the CPU, with no convolutions.
For the rest of the (non-image) datasets, it's already common knowledge that boosting methods are competitive with neural nets.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#50"In contrast to deep neural networks which require great effort in hyper-parameter tuning, gcForest is much easier to train." Hyperparameter tuning is not as much of an issue with deep neural networks anymore. Thanks to BatchNorm and more robust optimization algorithms, most of the time you can simply use Adam with a default learning rate of 0.001 and do pretty well. Dropout is not even necessary with many models tha…
GAN training is still spooky mysterious and can easily fail in nonintuitive ways.
Sometimes GANs converge or not depending on the random number seed, even with the same hyperparameters.