Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neural Networks: Zero to Hero

karpathy.ai

Post reply on HN
Print view View original

41–50 of 88 posts

Re: Neural Networks: Zero to Hero

#41

Post by amenhotep »

messe wrote: post #12
sh3rl0ck wrote: post #10

Earlier quoted context omitted.

Is it weird that I now know exactly which xkcd it will be just with conversational context? Granted I'm a bit of a Randall Munroe content addict, but it's become second nature now.

You're not alone. At this point I'm starting to recognise some by number as well.

A newly convicted criminal arrived in prison, and on the first night he was puzzled to hear his fellow inmates yelling numbers to each other. "36!" one would yell, and the rest would chuckle. "19!" went another, to uproarious laughter. "50," remarked a third wryly, which provoked groans and ironic cheers. Eventually his cellmate sat up and cried out "114" and it brought the house down.

In a lull, he asked his cellmate what on earth was going on? The cellmate explained that most of them had been in prison so long that they already knew all the jokes, so to save time they just referred to them by number. "Oh," says the man, "that makes sense. Can I try?"

His cellmate encouraged him to go ahead, so he stood up and went to the bars and shouted as loud as he could "95!"

Absolutely no reaction. His cellmate looked at him and shook his head. "You didn't tell it right."

Re: Neural Networks: Zero to Hero

#42

Post by 3abiton »

cube2222 wrote: post #25

I’ve gone through this series of videos earlier this year. In the past I’ve gone through many “educational resources” about deep neural networks - books, coursera courses (yeah, that one), a university class, the fastai course - but I don’t work with them at all in my day to day. This series of videos was by far the best, most “intuition building”, highest signal-to-noise ratio, and least “annoying” content to get th…

Karpathy has a great intuitive style, but sometimes it's too dumbed down. If you come from adjacent fields, it might be a bit dragging, but it's always entertaining

Re: Neural Networks: Zero to Hero

#46

Post by ronbenton »

3abiton wrote: post #42
cube2222 wrote: post #25

I’ve gone through this series of videos earlier this year. In the past I’ve gone through many “educational resources” about deep neural networks - books, coursera courses (yeah, that one), a university class, the fastai course - but I don’t work with them at all in my day to day. This series of videos was by far the best, most “intuition building”, highest signal-to-noise ratio, and least “annoying” content to get th…

Karpathy has a great intuitive style, but sometimes it's too dumbed down. If you come from adjacent fields, it might be a bit dragging, but it's always entertaining

>Karpathy has a great intuitive style, but sometimes it's too dumbed down

As someone who has tried some teaching in the past, it's basically impossible to teach to an audience with a wide array of experience and knowledge. I think you need to define your intended audience as narrowly as possible, teach them, and just accept that more knowledgeable folk may be bored and less knowledgeable folk may be lost.

Re: Neural Networks: Zero to Hero

#47

Post by a_r41 »

baxuz wrote: post #36

A bit of a tangential topic — what would you recommend to someone who wants to get into computer vision and 3D (NERFs, photogrammetry, 3DGS etc)? For someone who has a middling amount of math knowledge, what would you recommend? I went to uni 15y ago, but only had "proper" math in the first 2 semesters, let's says something akin to Calculus 1 and Linear Algebra 1. Hated math back then, plus I had horrible habits.

I've been working in the novel view synthesis domain since 2019 and I would recommend starting with "nerfstudio". The documentation does a good job of explaining all the components involved (from dataset to final learned representation), the code is readable and it's relatively simple to set up and run. I think it's a nice place to start from before diving deeper into the latest that is going on in the 3D space.

Re: Neural Networks: Zero to Hero

#48

Post by meken »

Has anyone gone through cs231n and this as well?

I went through the former and it was one of the best classes I’ve ever taken. But I’ve been procrastinating on going through this because it seems like there’s a lot of overlap and the benefit seems marginal (I guess transformers are covered here?).

Re: Neural Networks: Zero to Hero

#49

Post by jaccola »

baxuz wrote: post #36

A bit of a tangential topic — what would you recommend to someone who wants to get into computer vision and 3D (NERFs, photogrammetry, 3DGS etc)? For someone who has a middling amount of math knowledge, what would you recommend? I went to uni 15y ago, but only had "proper" math in the first 2 semesters, let's says something akin to Calculus 1 and Linear Algebra 1. Hated math back then, plus I had horrible habits.

For learning 3dgs (and its derivatives) I would recommend grabbing the original 3d Gaussian Splatting paper + repository and going through it and using an LLM to ask many questions.

LLMs aren't that great at explaining concepts a lot of the time so when you get stuck there, google around and learn that subtopic. E.g. you will come across "Jacobian" that you may or may not have seen before, but you can search Youtube and find a great Khan Academy/3b1b collab explaining it.

Get the code running also, play around with parameters, try to implement the whole thing from scratch, making sure you intuitively understand each part with the above method.

Obviously time scales vary for everyone, that having been said: I'd guess if you have a decent technical background, are OK feeling uncomfortable with the maths for a while (it is all understandable after a bit of pain), and are willing to keep plugging for a few hours a day you will have a very decent understanding in 6mo, and probably be "cutting edge" in a year or so (obviously the learning never ends, it is an active area of research after all!)

Re: Neural Networks: Zero to Hero

#50

Post by nobodyistaken »

lazarus01 wrote: post #45
nobodyistaken wrote: post #44

This is great, but if I'm starting ML from scratch, what would you recommend? I'm coming from a webdev background and have used LLMs but nothing about ML, might even need the refresher on math, I think.

https://deeplearningwithpython.io/

Is it wise to start to with deep learning without knowing machine learning?
Post reply on HN