Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: In 2022, what is the proper way to get into machine/deep learning?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 211 posts

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#51

Post by [deleted] »

wokwokwok wrote: post #30
santiagobasulto wrote: post #22

Can I suggest a longer, but (I think) better route? Try the Data/ML Engineer route. Instead of going directly into ML, try to work as a “supporter” of those doing ML. There’s a HUGE gap there, specially if you’re a good programmer. There are a lot of people in the “pure” ML space, people with science background, with phDs, etc. But there’s not enough people to support them: taking their models to producing, building…

> I think it can yield the highest success rate. At what? If you want to be an SRE for a data platform, sure, but this pretty thankless work: - cleaning up dodgy data - cleaning up behind low code data pipelines and other painful integration work with systems that suck and sometimes just don’t work (like PowerBI). - cleaning up behind data scientists that create models in arcane and imaginative ways and expect you to…

[deleted]

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#52

Post by urthor »

wokwokwok wrote: post #30
santiagobasulto wrote: post #22

Can I suggest a longer, but (I think) better route? Try the Data/ML Engineer route. Instead of going directly into ML, try to work as a “supporter” of those doing ML. There’s a HUGE gap there, specially if you’re a good programmer. There are a lot of people in the “pure” ML space, people with science background, with phDs, etc. But there’s not enough people to support them: taking their models to producing, building…

> I think it can yield the highest success rate. At what? If you want to be an SRE for a data platform, sure, but this pretty thankless work: - cleaning up dodgy data - cleaning up behind low code data pipelines and other painful integration work with systems that suck and sometimes just don’t work (like PowerBI). - cleaning up behind data scientists that create models in arcane and imaginative ways and expect you to…

Can confirm.

While data engineer is an excellent role for a fresh graduate, the data engineering profession shares many similarities with the SRE/IT professonional.

The best data engineers are the folk who had job title of DBAs of yonder year.

You will always be the supporting cast, rarely the star.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#53

Post by kqr »

alanfranz wrote: post #47

My 2c (not exhaustive for what you want to do, probably): 1) Get some statistics/probability basics. It's full of people (you can see a lot of analyses on Kaggle) that "do machine learning" but make very silly mistakes (e.g. turn categorical data into a float and use it as a continuous variable when training a model). 2) take a look at traditional machine learning approaches. Nowadays you're swamped by DL (a lot of g…

I second learning the statistics/probability basics.

Your first model should always be something that predicts a constant value, or maybe in really complicated cases something like a linear/logistic regression. Then you have a baseline to compare more advanced approaches to. But in order to understand how to use linear regression well, you need to understand how it works in the first place.

Also experiment structure, sampling design, hypothesis testing, etc. will tell you a lot about what conclusions you can and cannot draw from observational data, which is what a lot of ML is about.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#54

Post by NalNezumi »

I can only talk from my own experience but there's other way than just straight up go to ML/DL and crank out new models. That way is to come from the application domain side. [0]

For me that was robotics, with the motivation that traditional method felt like it wouldn't scale outside static environments so I started to look in to ML/DL (Deep Reinforcement Learning really) and from the looks of it I'm not alone. [0]

Now I do research in it, without a PhD nor taking any courses in it at my masters. (except one DL course where we had to code everything including gradient flow from scratch. No framework)

Frankly, going pure DL at research level today seems like a steep uphill; the top labs and research institute(including industries) are the ones that are producing (and notably training) most of the SOTA models. Getting in to those circles are your best bet but then a PhD at a top university under a top professor is the best bet, and competition to get in to those are insane

[0] https://www.natolambert.com/writing/path-into-ai

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#55

Post by mitchellgoffpc »

https://www.deeplearningbook.org/ and http://incompleteideas.net/book/the-book-2nd.html are excellent resources for supervised and reinforcement learning, respectively, and some knowledge of statistics and probability go a long way. But I think by far the most important thing is to just start training models, even very small ones, and developing an intuition for what works and what the failure modes are.

- Get really comfortable with matplotlib or your graphing library of choice. Plot your data in every way you can think of. Plot your models' outputs, find which samples they do best and worst on.

- Play around with different hyperparameters and data augmentation strategies and see how they affect training.

- Try implementing backprop by hand -- understanding the backward pass of the different layers is extremely helpful when debugging. I found Karpathy's CS231n lectures to be a great starting point for this.

- Eventually, you'll want to start reading papers. The seminal papers (alexnet, resnet, attention is all you need, etc) are a good place to start. I found https://www.youtube.com/c/YannicKilcher (especially the early videos) to be a very useful companion resource for this.

- Once you've read some papers and feel comfortable with the format, you'll want to try implementing something. Important tricks are often hidden away in the appendices, read them carefully!

- And above all, remember that machine learning is a dark art -- when your dataloader has a bug in its shuffling logic, or when your tensor shapes get broadcast incorrectly, your code often won't throw an error, your model will just be slightly worse and you'll never notice. Because of this, 90% of being a good ML researcher/engineer is writing tests and knowing how to track down bugs. http://karpathy.github.io/2019/04/25/recipe/ perfectly summarizes my feelings on this.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#56

Post by urthor »

psi75 wrote: post #44
wokwokwok wrote: post #30

Earlier quoted context omitted.

> I think it can yield the highest success rate. At what? If you want to be an SRE for a data platform, sure, but this pretty thankless work: - cleaning up dodgy data - cleaning up behind low code data pipelines and other painful integration work with systems that suck and sometimes just don’t work (like PowerBI). - cleaning up behind data scientists that create models in arcane and imaginative ways and expect you to…

This, 100%. That said, most data scientists don't do what you would consider real work (meaning, I assume, interesting work with significant mathematical/analytical meat). There just isn't a lot that's both interesting and useful to private-sector rent-seekers whose opinions of your work determine whether or not you advance. Most of the people doing real ML in industry are prestige hires--they're hired because their…

Strongly disagree.

There's a vast amount of work that doesn't involve unethical recommendation systems.

Expand your horizon outside the Bay Area.

The plurality of work I see is straightforward computer vision/NLP applications.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#57

Post by m3at »

There's so much resources it's hard to pick… If you're already studying CS and like to learn by building, I'd recommend to do that for ML too!

Pick a real problem, try to build a ML solution for it and while doing so keep a list of things you'd like to dig deeper into. Then go back to that list and pick one item to study, and iterate.

Happy to have a chat and give you specific pointers if you'd like (email in profile), I got my master in ML in 2016 and applied it in the industry since.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#58

Post by cassepipe »

psi75 wrote: post #44
wokwokwok wrote: post #30

Earlier quoted context omitted.

> I think it can yield the highest success rate. At what? If you want to be an SRE for a data platform, sure, but this pretty thankless work: - cleaning up dodgy data - cleaning up behind low code data pipelines and other painful integration work with systems that suck and sometimes just don’t work (like PowerBI). - cleaning up behind data scientists that create models in arcane and imaginative ways and expect you to…

This, 100%. That said, most data scientists don't do what you would consider real work (meaning, I assume, interesting work with significant mathematical/analytical meat). There just isn't a lot that's both interesting and useful to private-sector rent-seekers whose opinions of your work determine whether or not you advance. Most of the people doing real ML in industry are prestige hires--they're hired because their…

I really like your style. I think you should write a book (this is not sarcasm)

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#59

Post by natch »

wokwokwok wrote: post #35
robertlagrant wrote: post #32

Earlier quoted context omitted.

> It’s IT support for the people doing real work. This is an appalling perspective. Good MLE skills seem a lot harder to find that good ML ones.

Someone who can engineer infrastructure, pipelines and fire fight production issues is hard to find, but that’s not the point I was making. My apologies; It is real work; the point I was making is it’s not ML work , any more than writing a yaml file is ML work. If you want to write yaml files, any number of possibilities exist. If you want to work with machine learning, then don’t become a data engineer. The skills a…

With the right mindset it can be insanely fun building infrastructure, automating things, and engineering solutions such that improvements ratchet forward and fires get put out before they have a chance to grow. While the ML people, bless them, are chasing an 0.001 improvement in the metric of the day, data engineers can be having huge impact and changing the game. Meantime ML is becoming commoditized in its most common use cases.
Post reply on HN