Live data from Hacker News

Neural Networks: Zero to Hero

karpathy.ai

51–60 of 71 posts

Re: Neural Networks: Zero to Hero

#51

I am always lost in these blogs. Is there any gradual progression of understanding/exercises that one can follow to apply ML/NN/DL practically? Having these disconnected pieces of information with no clear link to one another feels like a lot of noise to me.

I’m almost done with both of the Andrew Ng coursera specializations they are the not at all disconnected or unclear. I don’t think I’ve ever learned so much so quickly in fact.

Re: Neural Networks: Zero to Hero

#52
post #50

Earlier quoted context omitted.

> his A100 GPU (a $10,000 GPU) These are available to rent per hour at much lower costs. The author mentions this in the video description.

Approximately 40 cents USD for 15 minutes from cursory research.

I'm completely unfamiliar with this market. Do you rent these on AWS? Or where?

Re: Neural Networks: Zero to Hero

#53
post #50

Earlier quoted context omitted.

Approximately 40 cents USD for 15 minutes from cursory research.

I'm completely unfamiliar with this market. Do you rent these on AWS? Or where?

https://jarvislabs.ai/pricing/

$1.29 per hour for a 40gb a100 apparently

https://lambdalabs.com/service/gpu-cloud#pricing

$1.10 per hour

Re: Neural Networks: Zero to Hero

#55

This was the first time I actually grokked backpropagation, just the first video alone is more lucid and valuable than any other resource about machine learning I had seen before, in fact it's so well explained that i managed to implement the library almost completely from memory after watching it - I cannot recommend it highly enough, especially for programmers without a math background! The only aspect I could see…

I find it genuinely stunning how ugly python gets in these projects. Almost immediately, even in a toy project (not tinygrad which is deliberately super dense).

Re: Neural Networks: Zero to Hero

#57

Earlier quoted context omitted.

> his A100 GPU (a $10,000 GPU) These are available to rent per hour at much lower costs. The author mentions this in the video description.

True, as much as I enjoy owning and controlling my own hardware, buying an A100 and then letting it sit idle while I procrastinate and play video games probably isn't the best use of resources. He did say "my GPU" (or similar) at one point, and I thought maybe he does enough ML stuff that he bought his own.

If you have an NVIDIA gaming GPU you can train reasonable transformers.

Re: Neural Networks: Zero to Hero

#58

Earlier quoted context omitted.

Having not watched the series, I can only assume he means logit as in a probability function from 0 to 1 https://deepai.org/machine-learning-glossary-and-terms/logit... .

logit is not a "probability function", quite the opposite. You can see this in the image in the link you posted (the x-axis is from 0-1, the y-axis is from -inf to inf). It transforms probabilities into log odds which is a linear space, and make combining probabilities much nicer. The inverse logit or logistic function takes log odds and transforms them back into probabilities. Most machine learning relies heavily on…

probability-related* function

Re: Neural Networks: Zero to Hero

#60

This was the first time I actually grokked backpropagation, just the first video alone is more lucid and valuable than any other resource about machine learning I had seen before, in fact it's so well explained that i managed to implement the library almost completely from memory after watching it - I cannot recommend it highly enough, especially for programmers without a math background! The only aspect I could see…

I agree. I'm looking forward to re-watching it to as it was so information dense.
Post reply on HN