Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: In 2022, what is the proper way to get into machine/deep learning?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 211 posts

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#31

Post by shoo »

fxtentacle wrote: post #10

Follow the HuggingFace Colab notebooks. They are well-written and language-related AIs are a great way to get started because you'll naturally have a feeling for what it should produce. Afterwards, do a statistics class. Most algorithms these days are based on softmax, meaning the cross-entropy between two discrete/continuous probability distributions. There's a lot of choice in which distribution to use to model wha…

> to train a state of the art ASR AI, you need roughly 100x A100 for a month, 100,000+ hours of audio recordings, and math knowledge to find a maximum likelihood path through a logit matrix.

Do you think there is a distinction between the kinds of problems that take some kind of "raw" signal data (audio, images etc) as input, where deep learning approaches appear to be fruitfully applied, and other kinds of problems that appear in many places in business and the public sector where the input space is not some kind of raw signal data but instead tabular data.

I have heard some people argue that the latter kinds of tabular-data style problems can be effectively tackled with a variety of statistical methods, and that deep learning style approaches do not offer an advantage.

Maybe fast.ai is commenting on the latter class of problem.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#32

Post by robertlagrant »

wokwokwok wrote: post #30
santiagobasulto wrote: post #22

Can I suggest a longer, but (I think) better route? Try the Data/ML Engineer route. Instead of going directly into ML, try to work as a “supporter” of those doing ML. There’s a HUGE gap there, specially if you’re a good programmer. There are a lot of people in the “pure” ML space, people with science background, with phDs, etc. But there’s not enough people to support them: taking their models to producing, building…

> I think it can yield the highest success rate. At what? If you want to be an SRE for a data platform, sure, but this pretty thankless work: - cleaning up dodgy data - cleaning up behind low code data pipelines and other painful integration work with systems that suck and sometimes just don’t work (like PowerBI). - cleaning up behind data scientists that create models in arcane and imaginative ways and expect you to…

> It’s IT support for the people doing real work.

This is an appalling perspective. Good MLE skills seem a lot harder to find that good ML ones.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#34

Post by LeetHacks »

wokwokwok wrote: post #30
santiagobasulto wrote: post #22

Can I suggest a longer, but (I think) better route? Try the Data/ML Engineer route. Instead of going directly into ML, try to work as a “supporter” of those doing ML. There’s a HUGE gap there, specially if you’re a good programmer. There are a lot of people in the “pure” ML space, people with science background, with phDs, etc. But there’s not enough people to support them: taking their models to producing, building…

> I think it can yield the highest success rate. At what? If you want to be an SRE for a data platform, sure, but this pretty thankless work: - cleaning up dodgy data - cleaning up behind low code data pipelines and other painful integration work with systems that suck and sometimes just don’t work (like PowerBI). - cleaning up behind data scientists that create models in arcane and imaginative ways and expect you to…

It highly depends. I was hired for a small research group that didn't have a product in production. Got hired for programming, was on the table discussing and contributing to research within a couple months without any background in ML.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#35

Post by wokwokwok »

robertlagrant wrote: post #32
wokwokwok wrote: post #30

Earlier quoted context omitted.

> I think it can yield the highest success rate. At what? If you want to be an SRE for a data platform, sure, but this pretty thankless work: - cleaning up dodgy data - cleaning up behind low code data pipelines and other painful integration work with systems that suck and sometimes just don’t work (like PowerBI). - cleaning up behind data scientists that create models in arcane and imaginative ways and expect you to…

> It’s IT support for the people doing real work. This is an appalling perspective. Good MLE skills seem a lot harder to find that good ML ones.

Someone who can engineer infrastructure, pipelines and fire fight production issues is hard to find, but that’s not the point I was making.

My apologies; It is real work; the point I was making is it’s not ML work, any more than writing a yaml file is ML work.

If you want to write yaml files, any number of possibilities exist.

If you want to work with machine learning, then don’t become a data engineer. The skills are, mostly, not related ML, and more closely aligned with SRE / devops.

It’s not infrastructure and helping build models as you mature and advance: it’s almost literally just infrastructure and fire fighting… in my, limited, 3 years of experience as such.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#36

Post by bitL »

santiagobasulto wrote: post #22

Can I suggest a longer, but (I think) better route? Try the Data/ML Engineer route. Instead of going directly into ML, try to work as a “supporter” of those doing ML. There’s a HUGE gap there, specially if you’re a good programmer. There are a lot of people in the “pure” ML space, people with science background, with phDs, etc. But there’s not enough people to support them: taking their models to producing, building…

Data Engineer is the outsourced part of what no ML researcher wants to do - a thankless, high-pressure, dead-end job which in no way leads to actually doing ML later - it would pigeon-hole the OP as unfit for real ML.

The best way is to take Stanford Deep Learning courses at SCPD, build a reputation, do real ML work (even if it's not a PhD, it's the same courses Stanford PhDs take).

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#37

Post by thrown_22 »

Learning how to do machine learning and learning how to clear data is a bit like asking what the best way to become an award winning author with very neat penmanship is.

One has nothing to do with the other.

That said data engineering at scale pays a lot more than deep learning but is also a lot less fun. Figure out which you'd rather do.

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#38

Post by umangkaushik »

shreyshnaccount wrote: post #5
gravelc wrote: post #2

Fastai. Updated series of lectures and notebooks for 2022. High level as well as building neural nets from scratch. Doing it at the moment and enjoying. Good as a starting point for more in depth studies.

part two of the course covers building everything from scratch and covers almost all of what OP wants, so this is a very good choice. anyone interested in a study group?

i am currently doing the 2019 part 2 of the course......a study group would be amazing

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#39

Post by phonebucket »

> Read research publications and try to implement them.

In my opinion, jump straight into this! Learn prerequisites as you need them.

I found Goodfellow's book [1] to be helpful to learn some basics.

But don't think you need to read the whole book before you start reading and implementing research papers.

If you try and build up all the fundamentals thorougly, you run the risk of going down a very deep rabbit hole e.g. learning real analysis so you can learn measure theory so you can learn measure theoretic probability theory so you can learn stats properly etc.

You can be a productive researcher and patch up the fundamentals over time.

[1] https://www.deeplearningbook.org/

Re: Ask HN: In 2022, what is the proper way to get into machine/deep learning?

#40

Post by robga »

fxtentacle wrote: post #10

Follow the HuggingFace Colab notebooks. They are well-written and language-related AIs are a great way to get started because you'll naturally have a feeling for what it should produce. Afterwards, do a statistics class. Most algorithms these days are based on softmax, meaning the cross-entropy between two discrete/continuous probability distributions. There's a lot of choice in which distribution to use to model wha…

fast.ai openly declares it doesn't have complete pipelines for audio, especially ASR.

They're also very open that areas like ASR and RL take a ton of compute to replicate SOTA. This shouldn't be used to substantiate any sort of criticism of fast.ai the library since it is just wrappers on pytorch. Maybe on the course.fast.ai page they could make some sort of disclaimer that some areas like ASR and yet to achieve the quantum leap of performance that we've seen in other domains.

I disagree completely that fast.ai is for toy problems. It's a development tool that in my experience provides for more rapid iteration than starting in vanilla pytorch. Use it for toys, for challenges, for research, or in industry. You can always rewrite your solution in pytorch once stable if there is benefit in doing so.

I agree with others that fast.ai is a great place to start.

Post reply on HN