Build a Deep Learning Library
zekcrates.quarto.pub
Build a Deep Learning Library
1–10 of 17 posts
Re: Build a Deep Learning Library
#2This is cool! This summer I made something similar but in C++. The goal was to build an entire LLM, but I only got to neural networks. GitHub repo here: https://github.com/amitav-krishna/llm-from-scratch. I have a few blogs on this project on my website (https://amitav.net/building-lists.html, https://amitav.net/building-vectors.html, https://amitav.net/building-matrices.html (incomplete)). I hope to finish that series eventually, but some other projects have stolen the spotlight! It probably would have made more sense to write it in Python because I had no C++ experience.
Re: Build a Deep Learning Library
#3It's alright, but a C version would be even better to fully grasp the implementation details of tensors etc. Shelling out to numpy isn't particularly exciting.
Re: Build a Deep Learning Library
#4It's alright, but a C version would be even better to fully grasp the implementation details of tensors etc. Shelling out to numpy isn't particularly exciting.
I agree! What NumPy is doing is actually quite beautiful. I was thinking of writing a custom c++ backend for this thing. Lets see what happens this year.
Re: Build a Deep Learning Library
#5[flagged]
Re: Build a Deep Learning Library
#6Did something similar a while back [1], best way to learn neural nets and backprop. Just using Numpy also makes sure you get the math right without having to deal with higher level frameworks or c++ libraries.
Re: Build a Deep Learning Library
#7Did something similar a while back [1], best way to learn neural nets and backprop. Just using Numpy also makes sure you get the math right without having to deal with higher level frameworks or c++ libraries. [1] https://github.com/santinic/claudioflow
Its nice! Yeah a lot of the heavy lifting is done by Numpy.
Re: Build a Deep Learning Library
#8It's alright, but a C version would be even better to fully grasp the implementation details of tensors etc. Shelling out to numpy isn't particularly exciting.
I agree! What NumPy is doing is actually quite beautiful. I was thinking of writing a custom c++ backend for this thing. Lets see what happens this year.
If someone is interested in low level tensor implementation details they could benefit from a course/book “let’s build numpy in C”. No need to complicate DL library design discussion with that stuff.
Re: Build a Deep Learning Library
#9Isn't this what Karpathy does as well in the Zero to Hero lecture series on YT? I am sure this is great as well!
Re: Build a Deep Learning Library
#10This is good. Its well positioned for software engineers to understand DL stuff beyond the frameworks.