21–30 of 56 posts
Earlier quoted context omitted.
> just pick neural networks for unstructured data ... except that NNs require a large amount of training data.
Sure there are constraint and pros and cons, but still: pick a neural network for unstructured data. Can always unsupervised pretrain and fine tune on a tiny dataset.
Earlier quoted context omitted.
Yeah, it is a good first benchmark. But view interpretability as separate from accuracy. You can explain black box algorithms just fine these days. Logistic regression is high bias low variance. If you were talking about fairness bias, then resistance to bias comes from logreg being too dumb to recognize complex non-linear patterns. Not necessarily a pro.
Would you mind talking to how black box interpretability is becoming well known? I've seen Shapley values used for feature interpretation, but not sure what else is being done.
Earlier quoted context omitted.
Sure there are constraint and pros and cons, but still: pick a neural network for unstructured data. Can always unsupervised pretrain and fine tune on a tiny dataset.
What qualifies as unstructured data? Would you consider text content to be structured or unstructured? (e.g. for classification of documents)
Earlier quoted context omitted.
Yeah, it is a good first benchmark. But view interpretability as separate from accuracy. You can explain black box algorithms just fine these days. Logistic regression is high bias low variance. If you were talking about fairness bias, then resistance to bias comes from logreg being too dumb to recognize complex non-linear patterns. Not necessarily a pro.
Sorry, I misspoke - will edit. Was talking about resistance to overfitting. Which largely comes from logistic regression's assumption of a linear decision boundary. It's true surprisingly often in classification tasks, and, when it's not, you can usually model it just fine with interaction variables. With an ANN, your easiest defense against overfitting is to have great big heaping piles of training data. That's some…
All the more power to you if a solid simple logreg model (or even no ML at all) is your first deliverable.
The image for logistic regression is hilariously wrong. It shows the sigmoid as a decision boundary. Also dont get hung up about no free lunch theorem. That is a great result in computer science theory with little practical impact: just pick neural networks for unstructured and GBDT for structured data. For the vast majority of real-life problems (not all possible problems) these are the single best algorithms.
Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.
The image for logistic regression is hilariously wrong. It shows the sigmoid as a decision boundary. Also dont get hung up about no free lunch theorem. That is a great result in computer science theory with little practical impact: just pick neural networks for unstructured and GBDT for structured data. For the vast majority of real-life problems (not all possible problems) these are the single best algorithms.
Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.
I know it _seems_ that way, but there's a surprising amount of nuance there and I think we're both fooling and limiting ourselves by letting this idea fester.
For one, unlike linear regression, logistic regression estimates aren't collapsible, so you can NOT interpret them as "changing this input by X changes the output by Y". That's only true if your set of covariates is perfect, which is never true, though in practice this interpretation might not be _that_ far off.
Another issue I see is practitioners not being aware of scaled/unscaled estimates; I've seen real papers from AI groups use logistic regression estimates like feature importance rankings, but using estimates in the scale of the original features, and not understanding the distinction when confronted about it.
From a practical sense, I think practitioners are much better served using random forests as their initial exploratory models. Less effort for results that are in practice at least as good as a well-prepped logit. Plenty issues with feature importance there, but not any worse than with logistic regression.
Earlier quoted context omitted.
Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.
Off topic-ish, but what do you mean by ^H^H^H^H? I feel like it's a joke I don't get.
Earlier quoted context omitted.
Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.
Off topic-ish, but what do you mean by ^H^H^H^H? I feel like it's a joke I don't get.
The image for logistic regression is hilariously wrong. It shows the sigmoid as a decision boundary. Also dont get hung up about no free lunch theorem. That is a great result in computer science theory with little practical impact: just pick neural networks for unstructured and GBDT for structured data. For the vast majority of real-life problems (not all possible problems) these are the single best algorithms.
Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.
They maybe directionally interpretable, but that’s about it.
The image for logistic regression is hilariously wrong. It shows the sigmoid as a decision boundary. Also dont get hung up about no free lunch theorem. That is a great result in computer science theory with little practical impact: just pick neural networks for unstructured and GBDT for structured data. For the vast majority of real-life problems (not all possible problems) these are the single best algorithms.
Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.
As far as I can tell, the GP raised no objection to logistic regression, they simply noted that the illustration didn't actually illustrate logistic regression but something else.