Ask HN: Full-on machine learning for 2020, what are the best resources? • Hacker News Board

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#21

Post by throwlaplace » Tue, Dec 31, 2019, 9:29 PM UTC

boltzmannbrain wrote: post #19

throwlaplace wrote: post #16

Earlier quoted context omitted.

>No. You will not get beyond copy-paste level without being comfortable with ML foundations. but everyone else in here is hyping fastai, which is not just copy-paste but wrapped copy-paste at that (so you're not even learning pytorch).

Sure, go through the fastai material and maybe write a blog post about how you learned ML (read: DL) in a few months. What you really learned is copy-pasting code (as you mentioned) and some neural net tricks (like a good learning-rate to start SGD). How to learn ML? Do fastai + reading Daphne Koller's and Chris Bishop's books on PGMs + re-implementing a paper on Gaussian process classification + another paper on GNN…

bishop's book is a good suggestion (i prefer hastie) for ml but you have to admit that

1. fastai is neural nets 2. bishop's book (and whoever else's) are grad books that require considerable mathematical training to really profit from 3. the aforementioned books don't teach anything practical!

so ultimately i completely agree with the op of this thread - just jump in and read around when things don't work how you expect.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#22

Post by rb808 » Tue, Dec 31, 2019, 9:44 PM UTC

codingslave wrote: post #6

Honestly, skip all of the courses. Pick a problem to solve, start googling for common models that are used to solve the problem, then go on github, find code that solves that problem or a similar one. Download the code and start working with it, change it, experiment. All of the theory and such is mostly worthless, its too much to learn from scratch and you will probably use very little of it. There is so much ml cod…

When I've learnt something it often is helpful to get well known problems so you get to compare to how other solve it too. Kaggle was good for big data stuff like that. I'm not sure about ML.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#23

Post by rronalddas » Tue, Dec 31, 2019, 9:57 PM UTC

echelon wrote: post #8

codingslave wrote: post #6

Honestly, skip all of the courses. Pick a problem to solve, start googling for common models that are used to solve the problem, then go on github, find code that solves that problem or a similar one. Download the code and start working with it, change it, experiment. All of the theory and such is mostly worthless, its too much to learn from scratch and you will probably use very little of it. There is so much ml cod…

This. Learn top down, not bottom up. Watch maybe one or two short videos on back propagation. You don't need to be muddled in the theory and the math - you can become productive right away. Once you start playing with pytorch and tensorflow models (train them yourself or do transfer learning), you'll start to develop an intuition for how the network graphs fit together. You'll also pick up tools like tensorboard. Als…

> You do not need to be a data scientist. Anybody can do it. That said, a good GPU will help a lot. I'm using two 1080Ti in SLI and they're pretty decent.

You can also use Google colab for a free GPU/TPU

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#24

Post by eachro » Tue, Dec 31, 2019, 9:59 PM UTC

I think it depends on what you want to focus on. If you want to do deep learning, fast.ai is probably the best resource available. Jeremy Howard and Rachel Thomas (the two founders) have poured quite a lot into fostering a positive, supportive community around fast.ai which really does add quite a lot of value.

If you want to really understand the fundamentals of machine learning (deep learning is just one subset of ML!), there is no substitute for picking up one of the classic texts like: Elements of Statistical Learning (https://web.stanford.edu/~hastie/ElemStatLearn/), Machine Learning: A Probabalistic Approach (https://www.cs.ubc.ca/~murphyk/MLbook/) and going through it slowly.

I'd recommend a two pronged approach: dig into fast.ai while reading a chapter a week (or at w/e pace matches your schedule) of w/e ML textbook you end up choosing. Despite all of the hype of deep learning, you really can do some pretty sweet things (ex: classify images/text) with neural nets within a day or two of getting started. Machine learning is a broad field, and you'll find that you will never know as much as you think you should, and that's okay. The most important thing is to stick to a schedule and be consistent with your learning. Good luck on this journey :)

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#25

Post by sweeneyrod » Tue, Dec 31, 2019, 10:00 PM UTC

codingslave wrote: post #6

Honestly, skip all of the courses. Pick a problem to solve, start googling for common models that are used to solve the problem, then go on github, find code that solves that problem or a similar one. Download the code and start working with it, change it, experiment. All of the theory and such is mostly worthless, its too much to learn from scratch and you will probably use very little of it. There is so much ml cod…

I definitely agree that you don't need to go deep into theory to be able to do useful things. But I think the bias-variance tradeoff is a very bad example of "useless theory". It's essentially just another name for overfitting/underfitting, which are approximately the most important ML concepts there are.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#26

Post by blondin » Tue, Dec 31, 2019, 10:07 PM UTC

codingslave wrote: post #6

Honestly, skip all of the courses. Pick a problem to solve, start googling for common models that are used to solve the problem, then go on github, find code that solves that problem or a similar one. Download the code and start working with it, change it, experiment. All of the theory and such is mostly worthless, its too much to learn from scratch and you will probably use very little of it. There is so much ml cod…

kudos to OP! AI & ML are also on my list for 2020!!

> All of the theory and such is mostly worthless, its too much to learn from scratch and you will probably use very little of it.

i, too, believe in code before theory. but not for stats, artificial intelligence, or machine learning, numerical computing, etc. why?

because, for instance, if you compare a popular & successful machine learning framework to a "build your own deep neural network in 150 lines of python", the difference as far as data structures or programming constructs choices will be staggering.

especially if you are an experienced programmer. or just someone who cares about the data structures and programming constructs in the first place. but these choices are not accidental!

you will find that "parameters" are represented by a "class", ie. objects with associated operations and not values. why? because you want to do things like accumulate contributions to derivatives, and all these other calculus things i thought i was never going to ever use.

theory is important for people who truly care!

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#27

Post by codingslave » Tue, Dec 31, 2019, 10:09 PM UTC

sweeneyrod wrote: post #25

codingslave wrote: post #6

Honestly, skip all of the courses. Pick a problem to solve, start googling for common models that are used to solve the problem, then go on github, find code that solves that problem or a similar one. Download the code and start working with it, change it, experiment. All of the theory and such is mostly worthless, its too much to learn from scratch and you will probably use very little of it. There is so much ml cod…

I definitely agree that you don't need to go deep into theory to be able to do useful things. But I think the bias-variance tradeoff is a very bad example of "useless theory". It's essentially just another name for overfitting/underfitting, which are approximately the most important ML concepts there are.

I would again argue, the natural progression for this concept would be:

1.) Trains classifier 2.) My train error was so low! Why is my validation error so high 2.) Googles -> Why is my classifier training error lower than my validation error 3.) Learns about overfitting 4.) learns about bias variance

Its always a natural progression. Reading about this stuff without encountering it means it usually doesnt stick, and really doesnt make that much sense.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#28

Post by jdminhbg » Tue, Dec 31, 2019, 10:22 PM UTC

codingslave wrote: post #6

Honestly, skip all of the courses. Pick a problem to solve, start googling for common models that are used to solve the problem, then go on github, find code that solves that problem or a similar one. Download the code and start working with it, change it, experiment. All of the theory and such is mostly worthless, its too much to learn from scratch and you will probably use very little of it. There is so much ml cod…

For what it's worth, this is basically the learning model fast.ai works on. You start by just applying pre-built models to things, then learn how to tweak them, then learn the theory that makes the tweaks work.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#29

Post by blululu » Tue, Dec 31, 2019, 10:25 PM UTC

The answer to this question depends on your level of computer & math proficiency. Some folks here have been debating about the relative merits of practice vs. theoretical foundations, but this dispute makes some assumptions about where you are starting from and where you are most comfortable. The fastest way to learn something is to fit it into a framework that you already understand. If you have a PhD in theoretical physics/abstract mathematics (like a lot of ML researchers), then the more mathematical (theoretical) frameworks will be a good way to build deep intuitions. If, on the other hand, you are more into applied data analysis, then you will probably find that working on applications will be the easiest way to go.

Personally, I enjoyed both Andrew Ng's and Geoffrey Hinton's respective courses on ML and Neural Networks on Coursera. You may also want to check out Michael Neilsen's online essay on deep learning (http://neuralnetworksanddeeplearning.com). Ultimately I would also encourage you to supplement your understanding by applying this work to your own applications. The universe is often the best teacher.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#30

Post by 1996 » Tue, Dec 31, 2019, 10:31 PM UTC

Whoever read this - please please please ignore the posts that suggest to just play with numbers. This is the equivalent of suggesting to someone who wants to learn how to code to copy-paste formulas into excel. Just don't be that person.

To be very blunt, in 2020 most ML is still glorified statistics, except you lose the insights and explanations. The only tangible improvements can be random forests - some times. 99% of the stuff you can do with basic statistics. 99% of the coders I know just don't know statistics besides the mean (and even with that, they do senseless things like doing means of means)

So learn statistics - basic statistics, like in the "for dummies" book series.

If you want to be a little more practical, stats "for dummies" is often found in disciplines that depends on stats, but are not very good in math - biology, psychology, and economics are great candidates.

So just download biology basis stats (to know how to compare means - this gives you the A/B test superpower), then psychology factor analysis (to know PCA - this gives you the dimension reduction superpower) then econometrics basic regression (to know linear regression)

With these 3 superpowers, you will be able to do more than most of the "machine learning" people. When you have mastered that, try stuff like random forest, and see if you still think it's as cool as it's hyped to be.