31–40 of 56 posts
Um, this seems very fishy to do, right? What am I missing here?
No mention of Deep Learning in a top ML algorithm list in 2018? Kinda odd if you ask me. Also, in the SVM section, no mention of kernel methods? (yet the picture shows a windy boundary). Also odd.
Neural nets and SVM were competitors - competitively applicable and competitively difficult - in the aughts. Deep learning has now pulled away but not by the discovery of fundamentally more complicated methods. Rather, the process has involved lot and lots and lots of little refinements, through throwing lots of people, advanced-math intuitions and computing power at it, etc. Learning everything needed to create state-of-the-art results is hard (as far I can tell/guess) but the basics are reasonably simple.
The image for logistic regression is hilariously wrong. It shows the sigmoid as a decision boundary. Also dont get hung up about no free lunch theorem. That is a great result in computer science theory with little practical impact: just pick neural networks for unstructured and GBDT for structured data. For the vast majority of real-life problems (not all possible problems) these are the single best algorithms.
(Btw agree with your other comment.)
> The technique (linear discriminate analysis) assumes that the data has a Gaussian distribution (bell curve), so it is a good idea to remove outliers from your data before hand. Um, this seems very fishy to do, right? What am I missing here?
Outlier removal should be done much more carefully when the goal is inference; trying to test if your hypothesis is true. Here, outlier removal is a super easy way to accidentally p-hack. Current best-practice is to pre-register your analysis, including how you'll define and handle outliers.
For predictive goals, where the idea is to predict the class/value of unseen data, outlier removal is often a good way to keep your bias in check and not bias your model towards the outliers. The trade-off is that future outliers will be predicted as though they weren't, e.g. much closer to the mean than they should be. This is what the article is trying to do.
There's also a whole wide world of outlier & anomaly detection, where you want to say e.g. "this new data point is probably an outlier".
Earlier quoted context omitted.
Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.
> generates parameters that you can easily explain to someone with no background in math I know it _seems_ that way, but there's a surprising amount of nuance there and I think we're both fooling and limiting ourselves by letting this idea fester. For one, unlike linear regression, logistic regression estimates aren't collapsible, so you can NOT interpret them as "changing this input by X changes the output by Y". Th…
tl;dr: The upshot is that non-collapsibility means that I can't use LR coefficients for things that I don't really need to use them for, anyway. That doesn't feel like a crippling limitation to me.
(Well, also, I have to occasionally pause to cross my fingers and say, "ceteris paribus," under my breath, which does admittedly make people think I'm some sort of weird Harry Potter nut. Which is OK. They're not wrong, they're just right for the wrong reason.)
Nor does it render its coefficients less interpretable than those of most other models. "Less interpretable than OLS" can still be pretty darn interpretable.
I've been wondering, is nearest neighbour really ML? No part of the logic is learned from the data. It feels just like a glorified lookup table, where the look up is fuzzy to some predefined definition of nearest.
k-nearest neighbors classification is one of the first non-linear supervised learning algorithms. Its predictions are derived from the data sample distances. It is basically a glorified fuzzy lookup table, but then again, so can one view deep learning (fuzzy hierarchical localized lookup). Pure memorization can even outperform logistic regression, especially with big data sets, so there is some recent debate as to wh…
So in essence, the rules which relates input to output are learned from the data in deep learning.
In nearest neighbour, the rule wasn't learned, we figured out the rule ourself: "use the nearest data point's result".
But in deep learning, the rule might be something like when feature x and y are between z range of each other and etc. And this rule is not defined by us, but by the weights which are learned from the data.
Effectively, deep learning thus learns the rules that define the relationship between input and class. But nearest neighbor is just a static rule that happens to be pretty general in essence, so it gives okay result for a lot of problems.
Not an AI expert, so take all this as my simple current understanding.
I'm a machine learning newbie, and I'd really like to speak with someone in the field to figure out what I'm wanting to learn. The problem I have is that I have a large labeled data set of spoken words and phonemes from a single speaker. I'd like to train a model and generate new phonemes (of various pitches, speeds, and intonations) with which to build a concatenative speech engine. What algorithms and models would…
For the techniques, Wavenet and Tacotron (e.g. https://google.github.io/tacotron/publications/tacotron2/) seem to be the state of art, but they are reportedly hard to replicate.