It's fun, but it's cheating a bit if you use numpy. Can someone implement one with all the matrix math from scratch too?
shameless self-promotion: https://github.com/siekmanj/sieknet
Implementing a Neural Network from Scratch in Python
41–50 of 104 posts
Re: Implementing a Neural Network from Scratch in Python
#42Why are deep learning researchers allergic to meaningful variable names?
Because you'll look more impressive if fewer people quickly understand your code.
Re: Implementing a Neural Network from Scratch in Python
#43My first reading of this headline was "Implementing a Neural Network in Scratch". Now that I would like to see. https://scratch.mit.edu/
Here's one! https://scratch.mit.edu/projects/26682825/
Re: Implementing a Neural Network from Scratch in Python
#44Friendly comment: at the beginning of section 4 there's this sum:
1
∑(y_true - y_pred)^2
i=1
Re: Implementing a Neural Network from Scratch in Python
#45Why are deep learning researchers allergic to meaningful variable names?
I'm not sure what you mean in this case: the Python code is very well written and extremely well commented.
I think this is an extremely well written tutorial! Kudos to the author.
Re: Implementing a Neural Network from Scratch in Python
#46Why are deep learning researchers allergic to meaningful variable names?
I've been told that not knowing better is a typical reason (and another reply says so), but my anecdotal contribution is different.
When I'm implementing algorithm, I'll typically be implementing something that uses single letter variable names. It's easier to do with consistent variable names. That's it. I once tried meaningful variable names in LaTeX to make code nicer, but long variable names rendered as math are an unreadable mess, so I went back to short variable names.
That being said, it's not hard to change variable names with a good IDE, and in LaTeX you can alias variable names with \newcommand.
Re: Implementing a Neural Network from Scratch in Python
#47Runnable code from the article: https://repl.it/@vzhou842/An-Introduction-to-Neural-Networks
I see so many implementations of NeuralNets from Scratch on Repl.it (I'm a co-founder) but this is one of the simplest and clearest one.
One that blew my mind recently is one by a 12 year old programmer: https://repl.it/@pyelias/netlib
Re: Implementing a Neural Network from Scratch in Python
#48Friendly comment: at the beginning of section 4 there's this sum: 1 ∑(y_true - y_pred)^2 i=1 There's no 'i' in the sum, should probably be y_true_i - y_pred_i.
I do get that in the example given there's only one element in each of the true/predicted vectors, but it threw me off for a sec.
Re: Implementing a Neural Network from Scratch in Python
#49If you're interested in learning something like this, I highly recommend Udacity's deep learning nanodegree. They have a section that teaches you how to build your own neural network with the the help of numpy. They even have a section where you write your own sentimental analysis neural network from scratch. Easily the best explanation and notebook to follow I have seen so far. I've been taking their revamped one that focuses on PyTorch rather than Tensorflow.
Disclaimer: I am taking the deep learning nanodegree degree for free but its sponsored by Facebook rather than Udacity directly.
Re: Implementing a Neural Network from Scratch in Python
#50I appreciate the author showing some of the math that drives the optimization. As a reader, I'd suggest introducing SGD ahead of the PD calculations, because it would give a clearer motivation for the PDs.