Live data from Hacker News

The annotated PyTorch training loop

idlemachines.co.uk

11–20 of 20 posts

Re: The annotated PyTorch training loop

#11
post #5
post #4

The host website seems extremely interesting in general. Anyone here used it before?

The design is similar to what Claude generates. Makes me skeptical that all the content is AI-generated.

Most people here seem to think the content is high quality.

Do you disagree?

Re: The annotated PyTorch training loop

#13
post #11
post #5

Earlier quoted context omitted.

The design is similar to what Claude generates. Makes me skeptical that all the content is AI-generated.

Most people here seem to think the content is high quality. Do you disagree?

Uhhh apart from OP I don’t see anyone commenting on the content quality?

Also OP didn’t even praise the content. They said it “seems interesting.”

Re: The annotated PyTorch training loop

#14
post #6
post #5

Earlier quoted context omitted.

The design is similar to what Claude generates. Makes me skeptical that all the content is AI-generated.

This looks better than 90% of websites I visited. The palette is pleasant, content has the right density and spacing is very relaxed. If this is AI-generated without some clever prompt tuning, frontend as a profession is done.

It's surprising to say people complimenting the color palette on this site. It's almost unreadable on my (very nice) monitor but clearly it works for many readers.

Re: The annotated PyTorch training loop

#15
post #10
post #7

PyTorch already does a huge job at keeping DL not verbose. It captures the complexity to allow you to make a whole GPT model with it, running on your GPU, and keep the code readable. Yes, there are a lot of breaking points, but at some point, DL is supposed to be non trivial, it handles math concepts most people don’t understand. That is why Data Scientists have a job ;)

I mean partial derivatives aren't that complicated if you know normal derivatives, which most people do. And backpropagation isn't too difficult either. The value of PyTorch lies more in utilizing accelerators like GPUs while offering a nice abstraction. But you can build your own (inefficient) tensor library without too much effort as e.g. Andrej Karpathy has shown in his "NN zero to hero" youtube series.

I hope you don't actually believe that most people know derivatives.

Re: The annotated PyTorch training loop

#16
post #10

Earlier quoted context omitted.

I mean partial derivatives aren't that complicated if you know normal derivatives, which most people do. And backpropagation isn't too difficult either. The value of PyTorch lies more in utilizing accelerators like GPUs while offering a nice abstraction. But you can build your own (inefficient) tensor library without too much effort as e.g. Andrej Karpathy has shown in his "NN zero to hero" youtube series.

I hope you don't actually believe that most people know derivatives.

I would hope most people do. It's in pretty much every high school curriculum, isn't it?

Re: The annotated PyTorch training loop

#17

Earlier quoted context omitted.

I hope you don't actually believe that most people know derivatives.

I would hope most people do. It's in pretty much every high school curriculum, isn't it?

https://xkcd.com/2501/

Even if most people had calculus in high school (which is not a given), I'd expect pretty much anyone who did not go into STEM to promptly forget it, just like I forgot much from chemistry and biology after studying Computer Science.

Re: The annotated PyTorch training loop

#18

Earlier quoted context omitted.

I hope you don't actually believe that most people know derivatives.

I would hope most people do. It's in pretty much every high school curriculum, isn't it?

I attended one of the better (read: top ~200) public high schools in the US, and you could graduate without even taking trigonometry

Re: The annotated PyTorch training loop

#19
post #4

The host website seems extremely interesting in general. Anyone here used it before?

I think the methodology is interesting, but you can probably design a similar, and probably better loop for "diving" into a topic and going through examples than is presented.

I have to imagine they just asked Claude to dive into a topic and generate example problems along the way.

As for the content... a lot feels like knowledge overload and concepts are introduced without explanation or "why". It basically says "here is a training loop" and never answers my immediate question of "...for what?" It also introduces random concepts like setting the seed that don't look like they're even in the final training loop it provides.

As a comparison, Pytorch itself has docs that go over the entire training loop as well. And it explains why you're loading the data its loading. With examples that run instead of being scattered on.

Re: The annotated PyTorch training loop

#20

Earlier quoted context omitted.

I hope you don't actually believe that most people know derivatives.

I would hope most people do. It's in pretty much every high school curriculum, isn't it?

Not in the United States at least. It's only something you would need if you went on to college anyway.
Post reply on HN