Learning Machine Learning: A beginner's journey

Even I recently started with ML/DL but my approach is more theoretical way. I started with Andrew's course, but alongside doing Python Machine Learning textbook,while testing my self on Kaggle. I hope to build some interesting system soon. The only thing I am worried about is getting a full time job, which I think always require someone with 2+ year experience.

> So I am doubling down on ML/DL.

The amount of free resources now available for learning machine learning/deep learning nowadays is robust and easy to comprehend. (indeed, Andrew Ng's Coursera class is very good). And running running ML code is even easier, with libraries like Tensorflow/Theano to abstract the ML gruntwork (and Keras to abstract the abstraction!)

I suspect that there may be machine learning knowledge crash, where the basics are repeated endlessly, but there is less unique, real world application of the knowledge learned. I've seen many Internet testimonials saying how "I followed an online tutorial and now I can classify handwritten digits, AI is the future!" The meme that Kaggle competitions are a metric of practical ML skill encourages budding ML enthusiasts to look at minimizing log-loss or maximizing accuracy without considering time/cost tradeoffs, which doesn't reflect real-world constraints.

Unfortunately, many successful real world applications of ML/DL are the ones not being instructed in tutorials as they are trade secrets (this is the case with "big data" literature, to my frustration). OpenAI is a good step toward transparency in the field, but that won't stop the ML-trivializing "this program can play Pong, AI is the future!" thought pieces (https://news.ycombinator.com/item?id=13256962).

Pretty much this. In the real world you have to deal with

- Too few labeled / garbage labeled data (70000 digits? How about only 1000 complex class objects?)

- Obscure bugs in custom implementation (yeah my custom layer works and gradient is correct... or wait why it diverges after 10k iteration? hmm).

- Timing/RAM constraints (it should segment an image under 10ms on Jetson TX1, well, good luck with GoogLeNet)

A counterpoint: Deep learning is currently hyped, making you not consider other techniques that might work better, or are simpler and work just as good. Deep learning might have a limited scope and turn out to be a dead end for areas other than the ones already examined.

I think this "machine learning for hackers" approach is just not enough. Oftentimes, you do need a solid theoretical/mathematical background. Most people seems to approach ML like they approach programming tools or libraries - learn just enough to get job done and move on.

I was studying machine learning from Andrew Ng's CS229 (the class videos are online. I think they date from 2008 or hereabout). There is no way you can progress beyond lecture 2 (out of 20) without a solid probability background. A solid background in probability/statistics probably means a good first course in Probability or maybe the first five chapters of "Statistical Inference" by Cassias and Berger. Similarly, for SVM, you need a solid background in Linear Algebra and so on. You probably also need a background Linear Optimization. Here are the recommendations by Prof. Michael Jordan https://news.ycombinator.com/item?id=1055389

Not a lot of people want to dive in this much. They have got things to do and who cares about proofs anyway. The thinking goes like "Most of the mathematics is abstracted away by libraries like scikit-learn. Let's get shit done.". Well, I think a lot of competitive advantage of Google/Facebook in ML is because they have staffed their engineering with people who have studied these things for years (by PhD). Compare that to flipkart's recommendations.

However, I don't think this problem is unique to ML/Data Science. It is equally bad in "Distributed systems". Let's use Docker, that's the future!