21–30 of 46 posts
https://makeyourownneuralnetwork.blogspot.com/ https://www.amazon.com/dp/1530826608/ref=cm_sw_r_tw_dp_U_x_x...
First part goes into what a NN is and how backprop works, second part is an implementation in Python.
I can highly recommend Joel Grus’ live coding video. He creates a deep learning library only using numpy in an hour and it’s really fun to watch it all come together. https://youtu.be/o64FV-ez6Gw
Thanks for submitting, this is going to come in handy for my education! I have an assignment next week which requires creating a neural network, then substituting various optimizers in place of backpropagation and comparing performance over iterations. Have found that simple numpy based NN's are easier to examine and connect the changes with the theory, this guide looks to help further with this understanding!
Too often you see articles like this and they start with $ import a_whole_bunch_of_stuff Good to see that this is not the case here :) The fast.ai course has a similar exercise in the beginning, but you'll still import the weights from somewhere else. Their fast.ai v1 library has a very short implementation too (loading the MINIST example dataset and then using Resnet18): from fastai import * from fastai.data import…
I guess that there's always a software layer which may be considered "backbone". For people designing networks, all the tedious work of building the network is just plumbing, and they probably expect it to be automated from a formal description of the network.
Too often you see articles like this and they start with $ import a_whole_bunch_of_stuff Good to see that this is not the case here :) The fast.ai course has a similar exercise in the beginning, but you'll still import the weights from somewhere else. Their fast.ai v1 library has a very short implementation too (loading the MINIST example dataset and then using Resnet18): from fastai import * from fastai.data import…
How to make cake, by Russell Peters First, you get cake. Then you make it for 20 minutes. Then you have cake.
Too often you see articles like this and they start with $ import a_whole_bunch_of_stuff Good to see that this is not the case here :) The fast.ai course has a similar exercise in the beginning, but you'll still import the weights from somewhere else. Their fast.ai v1 library has a very short implementation too (loading the MINIST example dataset and then using Resnet18): from fastai import * from fastai.data import…
You always have to start somewhere. Do you want to code all the OS from the ground up, or do you have minimal expectations about the environment your software will run on? Do you want to look at all the tiny details, or do you wish to focus on a specific aspect of the problem? The article author could also give an explanation of linear algebra, but I guess he expects the reader to be familiar with this part of the pr…
The point is that you don't need any software layers at all to code up a basic neural network implementation. A programming language with basic floating-point operations is all you need. The algorithms are not complicated so even x86 Assembly is practical for this purpose if you're already experienced with it. So the "backbone" can simply be your favorite compiler.
> Do you want to code all the OS from the ground up
If it's a "make your OS" course then yes - a simple OS of course. If you want to become experienced in compilers, then writing a compiler from scratch for a simple language is mandatory for people who want to have solid fundamentals. It's not a coincidence that projects like these are common in Computer Science and Software Engineering courses.
Another thing to try is calculating backpropagation by hand, on paper, with a small NN. This is what my mentor said his final exam on NN in college involved.