Live data from Hacker News

Ask HN: How to get started with machine learning?

news.ycombinator.com

31–40 of 131 posts

Re: Ask HN: How to get started with machine learning?

#31
DON'T LEARN NEURAL NETWORKS FIRST.

Instead, learn decision trees and more importantly enough statistics so you aren't dangerous.

Do you know what the central limit theorem is and why it is important? Can you do 5-fold cross validation on a random forest model in your choice of tool?

Fine, now you are ready to do deep learning stuff.

The reason I say not to do neural networks first is because they aren't very effective with small amounts of data. When you are starting out you want to be able to iterate quickly and learn, not wait for hours for a NN to train and then be unsure why it isn't working.

Re: Ask HN: How to get started with machine learning?

#32
post #23

You should have the equivalent of an undergraduate degree in mathematical statistics (calculus, linear algebra, et al). It should take about 4 years of full time study to achieve that. Forget about the code part. It's the least difficult part.

I think that this is a horribly impractical advice, and I keep seeing it everywhere. With modern tools and frameworks you can start learning and applying what you know on practice almost immediately. Check out Keras and the book "Deep Learning with Python"[1]. They have enabled me to train my first ANN in 2 days, and get to the point of building a MNIST recognizer in a month(and I was reading it pretty slowly). Sure,…

If you don't understand how it works your won't understand how to optimize things, how to do error analysis, how to implement better features and weights out of the box, how to choose the right algorithm from the start, how to do good cross validation ...

Yes, you can take a library and implement it in 10 minutes, but then you're really not learning machine learning, are you?

I will argue you do not need four years of math by any stretch, though. The stumbling block will be notation more than anything else. Relatively basic calculus and linear algebra will suffice.

They were right about one thing: the code is the least important part.

Re: Ask HN: How to get started with machine learning?

#33
post #6

My favorite textbook: Elements of Statistical Learning by Hastie. It's free, too! If you don't understand something in the book, back up and learn the pre-reqs as needed. http://web.stanford.edu/~hastie/ElemStatLearn/printings/ESLI...

Or you can start on easy mode with Introduction to Stastical Learning by the same authors.

http://www-bcf.usc.edu/~gareth/ISL/

Re: Ask HN: How to get started with machine learning?

#34
It depends on what you really want to do in the future. Learning a framework could be useless if you don't know how to do correctly basic things as creating a train, test and validation set.

There are basic things I think you must know before jumping into a framework or int any specific algorithm. First thing you probably will have to do is to collect the data and clean it. In order to do this correctly you need some basic statistics. For example you need to know what is a gaussian distribution and collect samples in a way that are representative of your problem. Then you may need to clean the samples, remove outlines, complete blank data, etc. So it is basic you know some statistics to do this right.I have seem people with a lot of knowledge of tools than then they are not able to create a train/test/validation set correctly and the experiment is completely invalid from here no matter what you do next (http://stats.stackexchange.com/questions/152907/how-do-you-u..., https://www.youtube.com/watch?v=S06JpVoNaA0&feature=youtu.be ). You also need to know how are you going to test your results, so again you need to know how to use a statistical test (f-test, t-test). So first thing, jump into statistics to understand your data.

The next step I think is to know some common things in machine learning as the no free lunch theorem, curse of dimensionality, overfitting, feature selection, how to select the current metric to asses your model and common pitfalls. I think the only way to learn this is reading a lot about machine learning and making mistakes by your own. At least now you have some things to search in google an start learning.

The third step would be to understand some basic algorithms and get the feeling of the type of algorithms, so you know when a clustering algorithm is needed or your problem is related classification or with prediction. Sometimes a simple random forest algorithm or logistic regression is enough for your problem and you don't need to use tensorflow at all.

Once you know the landscape of the algorithms I think it is time to improve your maths skills and try to understand better how the algorithms works internally. You might not need to know how a deep network works completely, but you should understand how a neural network works and how backpropagation works. The same with algorithms as k-means, ID3, A*, montecarlo tree search or most popular algorithms that you are probably are going to use in day to day work. In any case you are going to need to learn some calculus and algebra. Vectors, matrix and differential equations are almost everywhere.

You would probably have seen some examples when learning all the stuff I talked about, then it is time to go to real examples. Go to kaggle and read some tutorials, read articles about how the community of kaggle has faced and winning the competitions. From here is just practice and read.

You can jump directly into a framework, learn to use it, have 99% accuracy in your test and 0% accuracy with real data. This is the most probably scenario if you skip the basic things of machine learning. I have seen people doing this and end up very frustrated because they don't understand how their awesome model with 99% accuracy doesn't work in the real world. I have also seen people using very complex things as tensorflow with problems that can be solved with linear regression. Machine learning is a very broad area and you need maths and statistics for sure. Learn a framework is useless if you don't understand how to use it and it might lead you to frustration.

Re: Ask HN: How to get started with machine learning?

#35
A good start in "classical" methods" (i.e.: before deep learning and convolutional neural networks) is the old standby, the Weka Data Mining library [1]. Along with the textbook, it will make you comfortable with methods like k-nearest neighbor, support vector machines, decision trees, and the like.

[1] http://www.cs.waikato.ac.nz/ml/weka/

Re: Ask HN: How to get started with machine learning?

#36
post #6

My favorite textbook: Elements of Statistical Learning by Hastie. It's free, too! If you don't understand something in the book, back up and learn the pre-reqs as needed. http://web.stanford.edu/~hastie/ElemStatLearn/printings/ESLI...

This book is great, but if your stats background isn't quite up to snuff, it can be an intimidating first-read.

Personally, I studied Duda & Hart's pattern recognition [1] and Casella & Berger's statistics text [2] simultaneously. This took about the equivalent of 2 semesters. Duda's text gets the main ideas across without being as heavy on the probability theory / stats.

Afterwards, I studied "Elements ..." by Hastie et al., which was far more readable after going through Casella & Berger's text. Now Hastie et al. is my go-to reference. I also should note that this all assumes that you also have the requisite math background: up to calc 3, linear algebra, and maybe some exposure to numerical methods (in particular, optimization).

[1]: https://books.google.com/books?id=Br33IRC3PkQC&lpg=PP1&pg=PR...

[2]: https://books.google.com/books/about/Statistical_Inference.h...

Re: Ask HN: How to get started with machine learning?

#37
post #31

DON'T LEARN NEURAL NETWORKS FIRST. Instead, learn decision trees and more importantly enough statistics so you aren't dangerous. Do you know what the central limit theorem is and why it is important? Can you do 5-fold cross validation on a random forest model in your choice of tool? Fine, now you are ready to do deep learning stuff. The reason I say not to do neural networks first is because they aren't very effectiv…

Thanks. Can you recommend any statistics books to be safe?

Re: Ask HN: How to get started with machine learning?

#38
To get intuition and the right foundation read Society of Mind. For me the book is more about thinking in terms of computation which is what (IMO) ML is about instead of statistics (of which is important to know too!).

Now practical: I think the best way to learn is pick an algorithm & representation and implement it in your favorite language. Bonus if you have your own language to work with.

I would start looking into Decision Trees first, implement them and then implement some use cases(, which follow after implementing them). Do this for other approaches, like ANN, which you can have it beat you at checkers which is strangely satisfying.

But keep in mind Minsky. I think he is like Archimedes doing "Calculus"-type approaches without fully realizing. Maybe you could be Newton?

Re: Ask HN: How to get started with machine learning?

#39

I'd be more interested in real life results on a small scale first. I too felt like ML is something new to try, but the lack of real world use cases on a small scale ( not google, Microsoft, ... ) Has kept me from trying/doing. I only saw the farm with image recognition for vegetables as an example for now. Anyone has other examples?

Captcha breaking - personally, I've just found it a very satisfying ML project...

Re: Ask HN: How to get started with machine learning?

#40
post #2

If you want to jump right in with "hello world" type TensorFlow (a tool for machine learning), see https://news.ycombinator.com/item?id=12465935 (how to fit a straight line using TensorFlow) If you like to study/read: the famous Coursera Andrew Ng machine learning course: https://www.coursera.org/learn/machine-learning If you just want course materials from UC Berkeley, here's their 101 course: https://news.ycombinat…

I actually recommend jumping right into the excellent Scikit-learn tutorials, http://scikit-learn.org/stable/tutorial/

Unlike some of the other complicated tools, sklearn is just a "pip install" away and includes all sorts of examples of different problems. Classification? Regression? Clustering? Representation learning? Perceptual embedding? Odds are, some part of sklearn covers all of that.

Post reply on HN