Live data from Hacker News

Ask HN: In 2022, what is the proper way to get into machine/deep learning?

news.ycombinator.com

1–10 of 211 posts

Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#1
By getting into machine or deep learning I mean building upto a stage to do ML/DL research. Applied research or core theory of ML/DL research. Ofcourse, the path to both will quite different.

Standing in 2022, what are the best resources for a CS student/decent programmer to get into the field of ML and DL on their own. Resources can be both books or public courses.

The target ability:

1. To understand the theory behind the algorithms

2. To implement an algorithm on a dataset of choice. (Data cleaning and management should also be learned)

3. Read research publications and try to implement them.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#5
post #2

Fastai. Updated series of lectures and notebooks for 2022. High level as well as building neural nets from scratch. Doing it at the moment and enjoying. Good as a starting point for more in depth studies.

part two of the course covers building everything from scratch and covers almost all of what OP wants, so this is a very good choice. anyone interested in a study group?

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#6
post #2

Fastai. Updated series of lectures and notebooks for 2022. High level as well as building neural nets from scratch. Doing it at the moment and enjoying. Good as a starting point for more in depth studies.

Is there any benefit in learning how to code the algo from scratch? In most cases, We will just use standard libraries, unless the goal is to build up intution?

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#8
post #2

Fastai. Updated series of lectures and notebooks for 2022. High level as well as building neural nets from scratch. Doing it at the moment and enjoying. Good as a starting point for more in depth studies.

part two of the course covers building everything from scratch and covers almost all of what OP wants, so this is a very good choice. anyone interested in a study group?

yes! Let's do it

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#9
post #2

Fastai. Updated series of lectures and notebooks for 2022. High level as well as building neural nets from scratch. Doing it at the moment and enjoying. Good as a starting point for more in depth studies.

Is there any benefit in learning how to code the algo from scratch? In most cases, We will just use standard libraries, unless the goal is to build up intution?

I think it's still helpful as it indeed will give you more understanding. You wouldn't want to go for optimised algo's etc unless that rocks your boat, but just basic implementations is good to learn the basics from imho. And pretty satisfying to boot, again, imho.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#10
Follow the HuggingFace Colab notebooks. They are well-written and language-related AIs are a great way to get started because you'll naturally have a feeling for what it should produce.

Afterwards, do a statistics class. Most algorithms these days are based on softmax, meaning the cross-entropy between two discrete/continuous probability distributions. There's a lot of choice in which distribution to use to model what and it will have strong effects on your gradients and, hence, training trajectory.

Concepts like shannon information and entropy are also very helpful for you to monitor training progress. Typical loss values will do exponential annealing and it'll be difficult to see further progress. But if you still reduce the bits of entropy in your classifier, learning is still going well. So you need to understand what to visualize and how to calculate that.

As for implementing research publications, maybe start with easy mode and go to paperswithcode.com . There, you will find papers AND their source code, so that you can look at how others implemented their paper.

As for FastAI and Kaggle, my personal impression is that it's mostly for toy problems. No real AI researcher would be willing to disclose their full source code to an international megacorp like H&M for a measly $15k in price money, yet similar terms appear to be the default on Kaggle:

https://www.kaggle.com/competitions/h-and-m-personalized-fas...

https://www.kaggle.com/competitions/dfl-bundesliga-data-shoo...

https://www.kaggle.com/competitions/feedback-prize-effective...

EDIT: Also, I strongly disagree with course.fast.ai on these points: "Myth (don’t need): Lots of math, Lots of data, Lots of expensive computers" To train a state of the art ASR AI, you need roughly 100x A100 for a month, 100,000+ hours of audio recordings, and math knowledge to find a maximum likelihood path through a logit matrix. Unless, of course, you're only working on toy problems.

Post reply on HN