11–20 of 40 posts
Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…
Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…
Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…
But in some fields like computer vision, we humans fail(at least till now) to make rules better than black box neural networks.
Also, there are another group of ML people deal with rules+data, I quite agree with this article about the two types of ML: http://projecteuclid.org/download/pdf_1/euclid.ss/1009213726...
Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…
For example if you just start trying lots of rules by hand on your fraud data set there is a good chance you'll come up with a rule that looks good on your data but doesn't generalize to new data.
The number of models (or rules or formulas) you try by hand increases this chance (this related to multiple testing) and worse the process that generates them isn't repeatable since one you know a feature worked on the data set you're biased towards finding it again.
So in ML you try to come up with a model generating process that is entirely automated and repeatable. This means you can do it repeatedly in cross validation, over bootstraps, out of time etc on the same data set and be fairly sure that it will generalize.
The goal can still be a simple rule or formula but you achieve this simplicity for simplicity by penalizing complexity (as in lasso) or doing explicit simplification (as in pruning).
The reason that complex "black box" models are so popular is that they often have really nice statistical properties in terms of generalization. It's fairly intuitive that averaging over a bunch of slightelly perterbed simple models will give you a nice combined model as in a random forest, gbm or other ensemble.
Deep neural networks are less intuitive but it's been hypothesized that the depth makes them less prone to overfitting than a shallow network.
A shallow network will have one global optimal set of weights that's simple and really good but it also has lots of locally optimal states that aren't as good...and the chances are high your training procedure (or manual search for a simple model that works) will get caught in one of these.
It's been shown that the for deeper networks these local optima tend to be much closer in performance to the global optima so in effect making the model more complex makes it less likely you'll end up with a bad model.
Earlier quoted context omitted.
You can order the standard machine learning texts from most to least math-y, and least to most modern: - Pattern Recognition and Machine Learning (Bishop 2007) - Machine Learning: A Probabilistic Perspective (Murphy 2012) - Deep Learning (Goodfellow, Bengio, Courville 2016) If you want cutting-edge material, read the Deep Learning book (which is still quite technical, though some of its content may be outdated in a f…
Where would you place "Elements of Statistical Learning" in relation to these, if you know?
As a researcher I tend to prefer the Bayesian perspective in Bishop, because it gives you a unifying framework for thinking about building your own models and learning algorithms. But lots of people seem to respect ESL and speak very highly about it. It's probably valuable if you are implementing one of the methods it covers and want to understand that specific method in great depth.
does anyone have recommendation on a text that is mathematically heavy but also looks at modern approaches/appleications?
You can order the standard machine learning texts from most to least math-y, and least to most modern: - Pattern Recognition and Machine Learning (Bishop 2007) - Machine Learning: A Probabilistic Perspective (Murphy 2012) - Deep Learning (Goodfellow, Bengio, Courville 2016) If you want cutting-edge material, read the Deep Learning book (which is still quite technical, though some of its content may be outdated in a f…
- Elements statistical Learning, Hastie et al
- Shalev-Shwartz and Ben-David: http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning...
- the late David MacKay's Info Theory
- Bayesian Reasoning in ML, Barber
- Hopcroft/Kannan (this is an older version, you can google latest: http://www.cs.cornell.edu/jeh/book112013.pdf
does anyone have recommendation on a text that is mathematically heavy but also looks at modern approaches/appleications?
You can order the standard machine learning texts from most to least math-y, and least to most modern: - Pattern Recognition and Machine Learning (Bishop 2007) - Machine Learning: A Probabilistic Perspective (Murphy 2012) - Deep Learning (Goodfellow, Bengio, Courville 2016) If you want cutting-edge material, read the Deep Learning book (which is still quite technical, though some of its content may be outdated in a f…
Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…
ML is about creating a process that produces models from data in a way that is likely to generalize to new data. For example if you just start trying lots of rules by hand on your fraud data set there is a good chance you'll come up with a rule that looks good on your data but doesn't generalize to new data. The number of models (or rules or formulas) you try by hand increases this chance (this related to multiple te…
Earlier quoted context omitted.
Where would you place "Elements of Statistical Learning" in relation to these, if you know?
You want ISLr ( http://www-bcf.usc.edu/~gareth/ISL/ ), not ESL. ESL is the prototype for the former.