Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Tour of the Top Algorithms for Machine Learning Newbies

towardsdatascience.com

Post reply on HN
Print view View original

51–56 of 56 posts

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#51

Post by claytonjy »

thanatropism wrote: post #44
claytonjy wrote: post #26

Earlier quoted context omitted.

> generates parameters that you can easily explain to someone with no background in math I know it _seems_ that way, but there's a surprising amount of nuance there and I think we're both fooling and limiting ourselves by letting this idea fester. For one, unlike linear regression, logistic regression estimates aren't collapsible, so you can NOT interpret them as "changing this input by X changes the output by Y". Th…

It's not that difficult to compute true marginal effects from logistic regression using something like the bootstrap (if you have a distribution for your coefficients) or explicit differentiation. Every traditional stats app (Stata, SAS, etc) has this.

What do you mean by "true" marginal effect? Are you suggesting a post-hoc procedure can correct estimates such that they are close enough to the estimate that would have been produced with a more complete model specification?

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#52

Post by claytonjy »

bunderbunder wrote: post #49
claytonjy wrote: post #41

Earlier quoted context omitted.

I had exactly that post in mind, it really raised my awareness of these issues. I agree with Jake's interpretation of the conditional interpretation of the estimates, but the practical issue is that virtually nobody not well-educated in statistics will do that correctly. In particular, people tend to do exactly what Jake concedes rarely makes any sense, which is comparing estimates across different model specificatio…

I guess it depends. I have the luxury of working in a very "this is machine learning, which is not to be confused with statistical inference" problem domain. It doesn't really even really make sense to interpret most the models I build as describing any sort of causal relationship, and when people are looking at the parameter estimates, they're really just trying to figure out, "What does this model think is importan…

That sounds nice!

Feature ranking seems like a clearly safe interpretation of betas, though I've been bitten too often by letting glm (in R) scale my predictors, giving me back estimates on the original scales, and thus incomparable, and seen it happen to others even more. Easy to miss when your original scales aren't all that different.

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#53

Post by jonathankoren »

thanatropism wrote: post #50
jonathankoren wrote: post #47

Earlier quoted context omitted.

Knowing there is a a probabilistic relationship expressed by the coefficients and saying that “do x and y will happen” isn’t the same thing.

Logistic regression essentially gives a conditional probability function, much like linear regression gives a conditional expectation function. You can compute log odds from logistic regression -- say, conditional to all other factors being left-handed makes you twice as likely to some binary effect. People were complaining that this isn't trivially done by staring at the coefficients, but people who can't think in p…

That's the problem. Log odds are not intuitive. Hell, even probabilities aren't intuitive, and that's much easier to think about. Look at all the people start crying that "it was wrong" when the less likely event happens when the prediction said 90% probability.

This isn't a crazy minority position "logistic regression is not interpretable" is truism from basic ML courses, and blog posts all over the internet.

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#54

Post by thanatropism »

jonathankoren wrote: post #53
thanatropism wrote: post #50

Earlier quoted context omitted.

Logistic regression essentially gives a conditional probability function, much like linear regression gives a conditional expectation function. You can compute log odds from logistic regression -- say, conditional to all other factors being left-handed makes you twice as likely to some binary effect. People were complaining that this isn't trivially done by staring at the coefficients, but people who can't think in p…

That's the problem. Log odds are not intuitive. Hell, even probabilities aren't intuitive, and that's much easier to think about. Look at all the people start crying that "it was wrong" when the less likely event happens when the prediction said 90% probability. This isn't a crazy minority position "logistic regression is not interpretable" is truism from basic ML courses, and blog posts all over the internet.

It's odd that "ML theory" (really data science theory) as proposed by blog posts would supersede established statistics.

Something is rotten in the kingdom of Denmark.

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#55

Post by jonathankoren »

thanatropism wrote: post #54
jonathankoren wrote: post #53

Earlier quoted context omitted.

That's the problem. Log odds are not intuitive. Hell, even probabilities aren't intuitive, and that's much easier to think about. Look at all the people start crying that "it was wrong" when the less likely event happens when the prediction said 90% probability. This isn't a crazy minority position "logistic regression is not interpretable" is truism from basic ML courses, and blog posts all over the internet.

It's odd that "ML theory" (really data science theory) as proposed by blog posts would supersede established statistics. Something is rotten in the kingdom of Denmark.

I think you’re just trolling now. More to the point, I think you fundamentally misunderstand what interpretability means.

Logistic regression as not being interpretable was drilled into me by one of the creators of AdaBoost in grad school. As I said, this is widely held position.

Post reply on HN