Live data from Hacker News

Practical Deep Learning for Coders 2022

fast.ai

31–40 of 86 posts

Re: Practical Deep Learning for Coders 2022

#31
post #10

This is awesome. One question I have always had - is the research on applying DL for images the most developed compared to other things? Even DL used for audio processing (classification, separation etc) seems to convert audio to spectral graphs and apply DL to that. Changing a problem to be expressed as image inputs will be an advantage when using DL as a solution. Would you agree ?

Working with a spectrogram is definitely similar to working with an image, and it's interesting to think why that's the case.

Take convolutional models, for example. Very effective for working with images because they're (a) parameter efficient, (b) learn local/spatial correlations in input features, and (c) exploit translational invariance. As an oversimplification, we can train models to visually identify "things" in images by their edges.

If you think about what's going on with an audio spectrogram, you can see the same concepts at work. There's local/spatial correlation - certain sounds tend to have similar power coefficients in similar frequency buckets. These are also correlated in time (because the pitch envelope of the word "yes" tends to have the same shape), and convolutional models can also exploit time-invariance (in the sense that convolutional models can learn the word "yes" from samples where the word appears with varying amounts of silence to the left and right).

That being said, the addition of the time domain makes audio quite hard to work with, and (usually) not as simple as just running a spectrogram through a vanilla image classification model. But it's definitely enlightening to think about how these models are "learning".

Re: Practical Deep Learning for Coders 2022

#32
post #3

Hi folks - I'm the creator/teacher of this course. I'd be happy to answer any questions that you have about the course, learning deep learning in general, or the state of deep learning in 2022.

How many hours do you think this course would take for an experienced developer with plenty of applied maths but ~no machine learning?

How easy is it to do the course on my own hardware rather than cloud notebooks? Would that make it closer to practical deployment?

Re: Practical Deep Learning for Coders 2022

#33
post #3

Hi folks - I'm the creator/teacher of this course. I'd be happy to answer any questions that you have about the course, learning deep learning in general, or the state of deep learning in 2022.

I hope you stay as humble as you have been. But you're my personal hero. It is just incredible what you have done for the world.

Re: Practical Deep Learning for Coders 2022

#34
post #30

There are too many poor design decisions in the fast.ai library. One should invest too much time just for the sake of learning the library's weird API, and then using it. Doing something custom is too difficult, in contrast to Jax, PyTorch, and even (poor library) TensorFlow. The coding practices are whimsical. The codebase wouldn’t pass code review in any respectable company. Variable namings are weird and super-pro…

[deleted]

Re: Practical Deep Learning for Coders 2022

#35
post #30

There are too many poor design decisions in the fast.ai library. One should invest too much time just for the sake of learning the library's weird API, and then using it. Doing something custom is too difficult, in contrast to Jax, PyTorch, and even (poor library) TensorFlow. The coding practices are whimsical. The codebase wouldn’t pass code review in any respectable company. Variable namings are weird and super-pro…

I deleted an earlier, angrier comment of mine.

Can you explain this last sentence (which I understand to be insulting and without basis): > Howard is great with one problem: he kinda hates math. It might also seem that he ends up promoting anti-intellectualism.

Re: Practical Deep Learning for Coders 2022

#36
post #32
post #3

Hi folks - I'm the creator/teacher of this course. I'd be happy to answer any questions that you have about the course, learning deep learning in general, or the state of deep learning in 2022.

How many hours do you think this course would take for an experienced developer with plenty of applied maths but ~no machine learning? How easy is it to do the course on my own hardware rather than cloud notebooks? Would that make it closer to practical deployment?

Not him nor will I talk about his course, but I’ve been in the field a reasonable amount of time (both on the academia and industry side). Honestly, applied maths will get you a long way and make it easier to digest the concepts (you might just see them as repackaged problems depending on your mileage). If you have good programming skills and discipline you practically have most of what you need.

Re the course, I just skimmed it and I think you can do most things on your own hardware but if you will actually use this for something practical (not just for you or a side project), being familiar with cloud tools is a big thing especially once you scale.

Re: Practical Deep Learning for Coders 2022

#37
post #28
post #26

Earlier quoted context omitted.

I tried the 2021 course but I didn't finish. I think the biggest friction for me was using the remote machine. I wasn't able to make steady progress like I do with my offline learning projects. How far away is the fast.ai from working on a Mac? PyTorch recently gained support ( https://pytorch.org/blog/introducing-accelerated-pytorch-tra... ) but that's only the start. Is this something that is being worked on?

The good news is that every lesson in this course is actually run on Kaggle Notebooks, which is a free cloud environments including GPUs. So you don't need to set up anything and it runs on any computer with a modern web browser! Mac support for all the libs used in the course will probably continue to improve in the coming months and there should be no reason you won't be able to run the stuff for the course locally…

Thanks Jeremy, I'll give it another go.

Re: Practical Deep Learning for Coders 2022

#38
post #3

Hi folks - I'm the creator/teacher of this course. I'd be happy to answer any questions that you have about the course, learning deep learning in general, or the state of deep learning in 2022.

Is there a new version of the book? All the links I find lead to the 2020 edition.

Re: Practical Deep Learning for Coders 2022

#40
post #30

There are too many poor design decisions in the fast.ai library. One should invest too much time just for the sake of learning the library's weird API, and then using it. Doing something custom is too difficult, in contrast to Jax, PyTorch, and even (poor library) TensorFlow. The coding practices are whimsical. The codebase wouldn’t pass code review in any respectable company. Variable namings are weird and super-pro…

I deleted an earlier, angrier comment of mine. Can you explain this last sentence (which I understand to be insulting and without basis): > Howard is great with one problem: he kinda hates math. It might also seem that he ends up promoting anti-intellectualism.

He says repeatedly "You don't need math", and stuff like that.

This is not insulting. That man is my hero, and I deeply respect him.

But his 2019/20 course was riddled with such statements. He repeatedly said that one doesn't need math, and showed tools like drawing math symbols on a website to learn their names and ride on that. No further math needed.

It's like you can wing it in Deep Learning without learning Math. His behavior throughout the course reinforced this attitude. It is harmful for new learners.

But I am fortunate that I didn't learn from that, but learned from some successful alumni example that Howard gave. One woman who was also a musician ('19/'20), she made it big, but Howard mentioned that she did the Ng course, and also read the Goodfellow book.

So, I took the cue, and did DL the proper way. Anybody I know in DL made it because they know the Math.

There are some influencer types in fastai community who has 10ks of followers and shills stuff and do media stuff. Other than that 1-2 people, everyone who made it in DL, did it because they knew the math.

So, I think that people might get the wrong idea hearing from Howard that "you don't need math".

This is one fault I find. It's not like I dislike him. I like the rest of him. I love his attitude on almost all other things. I love Jeremy Howard, and he is my hero.

Post reply on HN