Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep Learning Foundations to Stable Diffusion

course.fast.ai

Post reply on HN
Print view View original

11–20 of 121 posts

Re: Deep Learning Foundations to Stable Diffusion

#11

Post by jph00 »

jamestimmins wrote: post #9
jph00 wrote: post #4

Hi folks. Nice to see our new free (and ad-free) course here on HN! This course is for folks that are already comfortable training neural nets and understand the basic ideas of SGD, cross-entropy, embeddings, etc. It will help you both understand these foundations more deeply, since you'll be creating everything from scratch (i.e. from only Python and its standard library), and to understand modern generative modelin…

Thanks Jeremy, this looks awesome. Do you think there's much value in learning much of the math in addition to the material in these courses?

The math is covered in this course. We look at both ordinary and stochastic differential equations, and at Langevin dynamics / score matching, as well as the needed probability foundations. We only cover the necessary bits of these topics -- just enough to understand how diffusion models work (e.g. we don't look at the physics stuff which is where Langevin dynamics came from).

On the whole, we cover the needed math at the point where we use it in the course. But there's also a bonus lesson that focuses on just the mathematical side, if you're interested: https://youtu.be/mYpjmM7O-30

Re: Deep Learning Foundations to Stable Diffusion

#12

Post by detrites »

jph00 wrote: post #4

Hi folks. Nice to see our new free (and ad-free) course here on HN! This course is for folks that are already comfortable training neural nets and understand the basic ideas of SGD, cross-entropy, embeddings, etc. It will help you both understand these foundations more deeply, since you'll be creating everything from scratch (i.e. from only Python and its standard library), and to understand modern generative modelin…

> since you'll be creating everything from scratch (i.e. from only Python and its standard library)

Is this correct? I skimmed the notes for both parts 1 & 2 and they stated requiring PyTorch?

Re: Deep Learning Foundations to Stable Diffusion

#13

Post by jstx1 »

nbzso wrote: post #10

This is cool, but I have a fundamental question. Why learn machine learning when all the machines will learn how to create and control themselves and powerful elites will offload the burden of governments to control us? Or we race ourselves to fulfill the last jobs on earth? Is this progress for humanity or enslavement and the end of our species? I am serious. This question is honest. Maybe my IQ is too low to unders…

This question isn't specific to machine learning - if you've given up already, don't bother learning anything and see how it goes.

Re: Deep Learning Foundations to Stable Diffusion

#14

Post by jph00 »

detrites wrote: post #12
jph00 wrote: post #4

Hi folks. Nice to see our new free (and ad-free) course here on HN! This course is for folks that are already comfortable training neural nets and understand the basic ideas of SGD, cross-entropy, embeddings, etc. It will help you both understand these foundations more deeply, since you'll be creating everything from scratch (i.e. from only Python and its standard library), and to understand modern generative modelin…

> since you'll be creating everything from scratch (i.e. from only Python and its standard library) Is this correct? I skimmed the notes for both parts 1 & 2 and they stated requiring PyTorch?

Yes, after we create something from scratch, we then see how to use implementations from libraries such as PyTorch. For basic operations like convolution, whilst we do create a GPU-accelerated version, it's not as fast as PyTorch/cudnn's highly-optimised version, so we use the PyTorch implementation for the rest of the course, rather than our handmade version.

So that means that nothing is mysterious, since we know how it's all made, but we also see where and how to use existing libraries as appropriate.

In practice, during the course we end up using PyTorch for stuff like gradients, matrix multiplication, and convolutions, and our own implementations for a lot of the stuff that's at a higher level than that (e.g. we use our own ResNet, U-net, etc.)

Re: Deep Learning Foundations to Stable Diffusion

#15

Post by detrites »

jph00 wrote: post #14
detrites wrote: post #12

Earlier quoted context omitted.

> since you'll be creating everything from scratch (i.e. from only Python and its standard library) Is this correct? I skimmed the notes for both parts 1 & 2 and they stated requiring PyTorch?

Yes, after we create something from scratch, we then see how to use implementations from libraries such as PyTorch. For basic operations like convolution, whilst we do create a GPU-accelerated version, it's not as fast as PyTorch/cudnn's highly-optimised version, so we use the PyTorch implementation for the rest of the course, rather than our handmade version. So that means that nothing is mysterious, since we know h…

Amazing, that's exactly what I've been looking for and what I'm sure many here will appreciate. Thank you!

Re: Deep Learning Foundations to Stable Diffusion

#17

Post by nbzso »

jstx1 wrote: post #13
nbzso wrote: post #10

This is cool, but I have a fundamental question. Why learn machine learning when all the machines will learn how to create and control themselves and powerful elites will offload the burden of governments to control us? Or we race ourselves to fulfill the last jobs on earth? Is this progress for humanity or enslavement and the end of our species? I am serious. This question is honest. Maybe my IQ is too low to unders…

This question isn't specific to machine learning - if you've given up already, don't bother learning anything and see how it goes.

I never give up. Just asked for information, to fight my own bias. Obviously there is a world outside which will have analogue and mechanical parts. But this is not the question.

The question is why to cut your wrists with a blunt object and stream the event for validation is suddenly a rational behavior?

Re: Deep Learning Foundations to Stable Diffusion

#18

Post by andrewm4894 »

nbzso wrote: post #17
jstx1 wrote: post #13

Earlier quoted context omitted.

This question isn't specific to machine learning - if you've given up already, don't bother learning anything and see how it goes.

I never give up. Just asked for information, to fight my own bias. Obviously there is a world outside which will have analogue and mechanical parts. But this is not the question. The question is why to cut your wrists with a blunt object and stream the event for validation is suddenly a rational behavior?

Are you a LLM or something? :)

Re: Deep Learning Foundations to Stable Diffusion

#19

Post by jamestimmins »

jph00 wrote: post #11
jamestimmins wrote: post #9

Earlier quoted context omitted.

Thanks Jeremy, this looks awesome. Do you think there's much value in learning much of the math in addition to the material in these courses?

The math is covered in this course. We look at both ordinary and stochastic differential equations, and at Langevin dynamics / score matching, as well as the needed probability foundations. We only cover the necessary bits of these topics -- just enough to understand how diffusion models work (e.g. we don't look at the physics stuff which is where Langevin dynamics came from). On the whole, we cover the needed math a…

Excellent, thanks for the thorough response!

Re: Deep Learning Foundations to Stable Diffusion

#20

Post by nbzso »

andrewm4894 wrote: post #18
nbzso wrote: post #17

Earlier quoted context omitted.

I never give up. Just asked for information, to fight my own bias. Obviously there is a world outside which will have analogue and mechanical parts. But this is not the question. The question is why to cut your wrists with a blunt object and stream the event for validation is suddenly a rational behavior?

Are you a LLM or something? :)

Yes. Like you.:)
Post reply on HN