Live data from Hacker News

Ask HN: Full-on machine learning for 2020, what are the best resources?

news.ycombinator.com

91–100 of 122 posts

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#91

Honestly, skip all of the courses. Pick a problem to solve, start googling for common models that are used to solve the problem, then go on github, find code that solves that problem or a similar one. Download the code and start working with it, change it, experiment. All of the theory and such is mostly worthless, its too much to learn from scratch and you will probably use very little of it. There is so much ml cod…

Without experience in ML it's often hard to know what problems are solvable, how to frame the problem, and to tell a good solution in Github from a bad one, etc.

If you want to go an applied route I'd suggest starting somewhere like Kaggle and looking through the competitions for ones vaguely similar to yours. They've done all the hard work of choosing a challenging but solvable problem, sourcing and splitting the data, and choosing a metric. You then can see what techniques actually work really well, and benchmark different approaches. Academic challenges like Imagenet or Coco are also good for this, but you'll have to work harder to find relevant resources.

Once you've done this a couple of times, you can start framing your own problems, collecting and annotating your own datasets, deploying and maintaining models.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#92
post #41

Good free resources: - MIT: Big Picture of Calculus - Harvard: Stats 110 - MIT: Matrix Methods in Data Analysis, Signal Processing, and Machine Learning If any of these seem too difficult - Khan Academy Precalculus (they also have Linear Algebra and Calculus material). This gives you a math foundation. Some books more specific to ML: - Foundations of Data Science - Blum et al. - Elements of Statistical Learning - Has…

If you like books and you want to deeply understand ML techniques I'd suggest jumping straight into "Introduction to Statistical Learning" and only learning calculus/stats/matrix methods (linear algebra) as you need them (you really don't need much from them in practice).

But it's ok to start using libraries and fitting models without understanding how they work deeply, and coming back to these books later (just make sure you come back; there's lots of useful ideas in them!) In which case I'd recommend some of the resources the parent doesn't recommend

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#93
post #87

In following order: 1. Michael Nielson's book: http://neuralnetworksanddeeplearning.com/ 2. Stanford CS231n course: http://cs231n.stanford.edu/ 3. DRL hands on book: https://www.amazon.com/Deep-Reinforcement-Learning-Hands-Q-n... After this churn through research papers or medium articles on conv net architecture surveys, batchnorm, LSTM, RNN, transformers, bert. Write lots of code, try things out.

This may make sense if you want to do image processing and deep reinforcement learning. But there are lots of other domains.

For tabular data (which is probably most relevant in Pharma, and probably the best place to start) Introduction to Statistical Learning by Hastie et al and Max Kuhn's Applied Predictive modelling cover a lot of the classical techniques.

For univariate time series forecasting "Forecasting Principles and Practice" is great.

For natural language processing foundations Jurafsky's Speech and Language Processing is broadly recommended; for cutting edge natural language processing Stanford's CS224n is great: http://web.stanford.edu/class/cs224n/

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#94
post #30

Whoever read this - please please please ignore the posts that suggest to just play with numbers. This is the equivalent of suggesting to someone who wants to learn how to code to copy-paste formulas into excel. Just don't be that person. To be very blunt, in 2020 most ML is still glorified statistics, except you lose the insights and explanations. The only tangible improvements can be random forests - some times. 99…

You can do pose estimation with basic statistics?

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#95

There is a question I have been asking for quite some time. It is known that Python is the language of choice when practicing ML. But, can similar results be achieved using Powershell? What makes Python superior to Powershell when making models for ML?

Technically you can do it in any language, but in software engineering we tend to stand on the shoulders of giants in order to get the job done on time.

A lot of original excellent data processing, statistical analysis, and ML libraries were built into Python and R, so all the deep learning stuff was built on top of those. R is somewhat harder to integrate into a production pipeline due to its typical reliance on something like RStudio, so Python ended up being the de facto standard as it is also well supported in cloud computing environments.

With TensorFlow API's being written for Swift, we might start to see Swift competing with Python.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#96
post #30

Whoever read this - please please please ignore the posts that suggest to just play with numbers. This is the equivalent of suggesting to someone who wants to learn how to code to copy-paste formulas into excel. Just don't be that person. To be very blunt, in 2020 most ML is still glorified statistics, except you lose the insights and explanations. The only tangible improvements can be random forests - some times. 99…

You can do pose estimation with basic statistics?

Many business data is tabular (possibly with time component), and if you are working with tabular data, the OP’s advice is sound.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#97
post #30

Whoever read this - please please please ignore the posts that suggest to just play with numbers. This is the equivalent of suggesting to someone who wants to learn how to code to copy-paste formulas into excel. Just don't be that person. To be very blunt, in 2020 most ML is still glorified statistics, except you lose the insights and explanations. The only tangible improvements can be random forests - some times. 99…

Given that many data people run across is tabular, I appreciate your advice about the importance of statistics. Also kudos for mentioning hypothesis testing (no one in this thread mentioned it). Lastly, I’d add that ML practitioners will gain a lot by listening to statisticians and economists on the issue of data quality, e.g. selection bias.

That said, I am not as cynical about “machine learning.” ML and “data science” brought the importance of prediction front and center, i.e. can you fit a model that accurately predict the target value given a previously seen input? This point is made by the recently published stats textbook Computer Age Statistical Inference (Efron and Hastie).

In some applications, it may be beneficial to choose black box models with high predictive accuracy, as the goal for these applications is prediction, not interpreting individual model coefficients.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#98

I was in the same boat in 2014. I went a more traditional route by getting a degree in statistics and doing as much machine learning as my professors could stand (they went from groaning about machine learning to downright giddy over those two years). I worked as a data scientist for an oil-and-gas firm, and now work as a machine learning engineer (same thing, basically) for a defense contractor. I’ve seen some reall…

This is the correct progression IMHO. I can tell you’ve been in industry because it mimics my experiences.

Always start with a simple model and see how far you can get. Most of the improvements I’ve seen comes from “working the data” anyway. You will be surprised how much you can improve model performance just by working the data, or improving the quality of the underlying data alone. Also simple models give you a “baseline”. What is the point of reaching for neural networks if you don’t have a baseline performance metric to compare against? XGBoost is a godsend. It trains extremely quickly and is surprisingly difficult to beat in practice.

As you say, constantly sharpen your saw with regards to probability theory and mathematics in general. There is simply no way around this in the long run.

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#99
post #41

Good free resources: - MIT: Big Picture of Calculus - Harvard: Stats 110 - MIT: Matrix Methods in Data Analysis, Signal Processing, and Machine Learning If any of these seem too difficult - Khan Academy Precalculus (they also have Linear Algebra and Calculus material). This gives you a math foundation. Some books more specific to ML: - Foundations of Data Science - Blum et al. - Elements of Statistical Learning - Has…

Why don't you recommend fast.ai and kaggle?

and Andrew Ng's ML course?

Re: Ask HN: Full-on machine learning for 2020, what are the best resources?

#100

Honestly, skip all of the courses. Pick a problem to solve, start googling for common models that are used to solve the problem, then go on github, find code that solves that problem or a similar one. Download the code and start working with it, change it, experiment. All of the theory and such is mostly worthless, its too much to learn from scratch and you will probably use very little of it. There is so much ml cod…

If you want to raise your salary from $10 to $20 per hour, playing with existing models is the way to go.

If you want to make serious money solving real problems, take the time to learn about automated differentiation and all the related mathematics about how gradients flow backwards through the network.

But like the coding slave (great nick BTW) said, first play a bit, then learn how it works. Image transformation GANs are a lot of fun.

Here's why the learning part will be crucial to differentiate you from all the clueless outsourced cheap labor:

Recently, there has been a load of new AI papers by so-called scientists on optical flow, and even the greatest new approaches using millions of parameters and costing hundreds of thousands of dollars to train still DO NOT reach the general level of quality that the 2004 census transform approach had.

Similarly, there have been high-profile papers where people randomly chained together TensorFlow operations to build their loss function, oblivious to the fact that some intermediate operations were not differentiable and, hence, their loss would never back-propagate. As a result, all of their claims had to be fraudulent because one could mathematically prove that their network was incapable of learning.

The larger AI competitions have by now limited the number of submissions that teams are allowed to make per week, simply to discourage people from trying to guess the test results when their AI doesn't work as it should.

Or consider the Uber pedestrian fatality where their neural network was overtrained ( = bad loss function ) to the point where it was unwilling to recognize bicycles at night.

And lastly, not knowing about gradient descent will just waste boatloads of money by 100x-ing your training time. Most stereo disparity and depth estimation AI papers use loss functions that only work on adjacent pixels. That means for a single correction to propagate to all pixels in a HD frame, you'll need 1920 iterations when only 1 could be sufficient.

You will find that my examples are all from autonomous driving. That's because here the discrepancy between GPU-powered brute force amateurs and skilled professionals is the most striking. German luxury cars have integrated lane-keeping, street sign recognition, and safety distance keeping for 10+ years, so for those tasks there are proven algorithms that work on a Pentium III in real time. And now there's lots of NVIDIA GPU kiddies trying to reinvent the wheel with limited success.

For your future employer, you having a firm grasp of how gradients work is the difference between mediocre and state of the art results, and between affordable and too expensive. So if there is one single AI skill that is both exhausting to learn and crucially important, it is differentiation and gradient flow.

Post reply on HN