Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Tour of the Top Algorithms for Machine Learning Newbies

towardsdatascience.com

Post reply on HN
Print view View original

11–20 of 56 posts

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#11

Post by polkapolka »

1024core wrote: post #7
polkapolka wrote: post #5

The image for logistic regression is hilariously wrong. It shows the sigmoid as a decision boundary. Also dont get hung up about no free lunch theorem. That is a great result in computer science theory with little practical impact: just pick neural networks for unstructured and GBDT for structured data. For the vast majority of real-life problems (not all possible problems) these are the single best algorithms.

> just pick neural networks for unstructured data ... except that NNs require a large amount of training data.

Sure there are constraint and pros and cons, but still: pick a neural network for unstructured data. Can always unsupervised pretrain and fine tune on a tiny dataset.

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#13

Post by yorwba »

1024core wrote: post #7
polkapolka wrote: post #5

The image for logistic regression is hilariously wrong. It shows the sigmoid as a decision boundary. Also dont get hung up about no free lunch theorem. That is a great result in computer science theory with little practical impact: just pick neural networks for unstructured and GBDT for structured data. For the vast majority of real-life problems (not all possible problems) these are the single best algorithms.

> just pick neural networks for unstructured data ... except that NNs require a large amount of training data.

General-purpose machine learning algorithms require a large amount of training data. There's no magic that can make accurate predictions without any data to base them on. If you don't have enough data, the information needs to come from somewhere else.

If you have a large amount of data on a similar problem, you can try transfer learning to learn shared properties and only fine-tune the domain-specific stuff on a smaller data set.

If you have no quantitative data, but know domain experts, you can build a custom model based on their advice, with fewer parameters that need to be fit to the data you do have.

But if you have so little data that you can't train a neural network, you can't be getting new data very frequently. It might be cheaper to just pay a human to look at it.

If you don't even have enough data for humans to work with, fancy machine learning isn't going to help you.

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#14

Post by polkapolka »

bunderbunder wrote: post #10
polkapolka wrote: post #5

The image for logistic regression is hilariously wrong. It shows the sigmoid as a decision boundary. Also dont get hung up about no free lunch theorem. That is a great result in computer science theory with little practical impact: just pick neural networks for unstructured and GBDT for structured data. For the vast majority of real-life problems (not all possible problems) these are the single best algorithms.

Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.

Yeah, it is a good first benchmark. But view interpretability as separate from accuracy. You can explain black box algorithms just fine these days.

Logistic regression is high bias low variance. If you were talking about fairness bias, then resistance to bias comes from logreg being too dumb to recognize complex non-linear patterns. Not necessarily a pro.

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#15

Post by bunderbunder »

polkapolka wrote: post #14
bunderbunder wrote: post #10

Earlier quoted context omitted.

Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.

Yeah, it is a good first benchmark. But view interpretability as separate from accuracy. You can explain black box algorithms just fine these days. Logistic regression is high bias low variance. If you were talking about fairness bias, then resistance to bias comes from logreg being too dumb to recognize complex non-linear patterns. Not necessarily a pro.

Sorry, I misspoke - will edit. Was talking about resistance to overfitting. Which largely comes from logistic regression's assumption of a linear decision boundary. It's true surprisingly often in classification tasks, and, when it's not, you can usually model it just fine with interaction variables.

With an ANN, your easiest defense against overfitting is to have great big heaping piles of training data. That's something that's hard to come by in many interesting situations.

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#18

Post by probably_wrong »

echelon wrote: post #2

I'm a machine learning newbie, and I'd really like to speak with someone in the field to figure out what I'm wanting to learn. The problem I have is that I have a large labeled data set of spoken words and phonemes from a single speaker. I'd like to train a model and generate new phonemes (of various pitches, speeds, and intonations) with which to build a concatenative speech engine. What algorithms and models would…

The simplest way would be downloading something like MaryTTS [1], read the documentation, and train your own voice model. It won't be perfect, but shouldn't be too hard.

The best results would probably be achieved by implementing DeepMind's WaveNet paper [2], but it might be too much for what you need.

I'm not really sure what to suggest in between those two. Some kind of convolutional NN, I guess?

[1] mary.dfki.de

[2] arxiv.org/abs/1609.03499

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#19

Post by tomrod »

polkapolka wrote: post #14
bunderbunder wrote: post #10

Earlier quoted context omitted.

Logistic regression isn't sexy, but it can still achieve near state-of-the-art results, is reasonably resistant to bias^H^H^H^H variance, and generates parameters that you can easily explain to someone with no background in math. There's a lot of value in all that. Especially if your deliverable is something that a business is going to use, and not just a Kaggle entry.

Yeah, it is a good first benchmark. But view interpretability as separate from accuracy. You can explain black box algorithms just fine these days. Logistic regression is high bias low variance. If you were talking about fairness bias, then resistance to bias comes from logreg being too dumb to recognize complex non-linear patterns. Not necessarily a pro.

Would you mind talking to how black box interpretability is becoming well known? I've seen Shapley values used for feature interpretation, but not sure what else is being done.

Re: A Tour of the Top Algorithms for Machine Learning Newbies

#20

Post by 5minbreak »

didibus wrote: post #16

I've been wondering, is nearest neighbour really ML? No part of the logic is learned from the data. It feels just like a glorified lookup table, where the look up is fuzzy to some predefined definition of nearest.

k-nearest neighbors classification is one of the first non-linear supervised learning algorithms. Its predictions are derived from the data sample distances.

It is basically a glorified fuzzy lookup table, but then again, so can one view deep learning (fuzzy hierarchical localized lookup).

Pure memorization can even outperform logistic regression, especially with big data sets, so there is some recent debate as to what degree models memorize and to what degree they generalize.

Post reply on HN