Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Build a Deep Learning Library

zekcrates.quarto.pub

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Build a Deep Learning Library

#11

Post by butanyways »

p1esk wrote: post #8
butanyways wrote: post #4

Earlier quoted context omitted.

I agree! What NumPy is doing is actually quite beautiful. I was thinking of writing a custom c++ backend for this thing. Lets see what happens this year.

If someone is interested in low level tensor implementation details they could benefit from a course/book “let’s build numpy in C”. No need to complicate DL library design discussion with that stuff.

Yes!!

Re: Build a Deep Learning Library

#12

Post by butanyways »

silentsea90 wrote: post #9

Isn't this what Karpathy does as well in the Zero to Hero lecture series on YT? I am sure this is great as well!

If you are asking about the "micrograd" video then yes a little bit. "micrograd" is for scalars and we use tensors in the book. If you are reading the book I would recommend to first complete the series or atleast the "micrograd" video.

Re: Build a Deep Learning Library

#15

Post by megadragon9 »

Thanks for sharing! It's inspiring to see more people "reinventing for insight" in the age of AI. This reminds me of my similar previous project a year ago when I built an entire PyTorch-style machine learning library [1] from scratch, using nothing but Python and NumPy. I started with a tiny autograd engine, then gradually created layer modules, optimizers, data loaders etc... I simply wanted to learn machine learning from first principles. Along the way I attempted to reproduce classical convnets [2] all the way to a toy GPT-2 [3] using the library I built. It definitely helped me understand how machine learning worked underneath the hood without all the fancy abstractions that PyTorch/TensorFlow provides. I eventually wrote a blog post [4] of this journey.

[1] https://github.com/workofart/ml-by-hand

[2] https://github.com/workofart/ml-by-hand/blob/main/examples/c...

[3] https://github.com/workofart/ml-by-hand/blob/main/examples/g...

[4] https://www.henrypan.com/blog/2025-02-06-ml-by-hand/

Re: Build a Deep Learning Library

#16

Post by RestartKernel »

megadragon9 wrote: post #15

Thanks for sharing! It's inspiring to see more people "reinventing for insight" in the age of AI. This reminds me of my similar previous project a year ago when I built an entire PyTorch-style machine learning library [1] from scratch, using nothing but Python and NumPy. I started with a tiny autograd engine, then gradually created layer modules, optimizers, data loaders etc... I simply wanted to learn machine learni…

During my Bachelor's, I wrote a small "immutable" algebraic machine learning library based on just NumPy. This made it easy to play around with combining weights by simply summing two networks by whatever operations are supported on normal NumPy arrays.

... turns out, this is only useful in some very specific scenarios, and it's probably not worth the extreme memory overhead.

Re: Build a Deep Learning Library

#17

Post by butanyways »

megadragon9 wrote: post #15

Thanks for sharing! It's inspiring to see more people "reinventing for insight" in the age of AI. This reminds me of my similar previous project a year ago when I built an entire PyTorch-style machine learning library [1] from scratch, using nothing but Python and NumPy. I started with a tiny autograd engine, then gradually created layer modules, optimizers, data loaders etc... I simply wanted to learn machine learni…

Yes i've read your blogposts way back then. Nice work with the gpt-2!!
Post reply on HN