Implementing a Neural Network from Scratch in Python
victorzhou.com
Implementing a Neural Network from Scratch in Python
1–10 of 104 posts
Re: Implementing a Neural Network from Scratch in Python
#2My first reading of this headline was "Implementing a Neural Network in Scratch". Now that I would like to see.
Re: Implementing a Neural Network from Scratch in Python
#3There are so many little details to remember when you implement a Neural Network from "scratch". Or, I suppose, even if you do not.
You know what would be a great contribution? An extensive set of unit tests, or even just problems with solutions. That way people can write their own implementations and test them. And even if a person were to implement the net in Pytorch of Tensorflow, they could test the work.
So there would be a matrix of weights, and a vector of input nodes, and the "answer" would be the output vector. Then there would be another "answer" which is the output with a particular activation function, and so on.
This library would just be there so people who are doing their own implementation can test their work.
As I said, a unit test would work too but then it would have to be language specific. Just the matrix and answer would be language agnostic.
For people who think: can't you just make up an example yourself using a sheet of paper or in excel.
Yes, for most purposes this is fine, but if you forget one little implementation detail of a three layer network with a ReLu, you really want an external way to check that.
Re: Implementing a Neural Network from Scratch in Python
#4This is a great write up! I'm in the process of writing a similar article, although I think for many non-programmers, neural networks can be useful through using algorithms that are already made to solve problems where people have the data but don't understand the math.
This article is good if you've been around some math and have a little coding experience, I think Neural Nets are even more accessible than the author is making them out to be however.
For example, using something like YOLO for object detection can be done even if you've never taken algebra or a programming class, and really just requires some configuration and training on a dataset you have.
Re: Implementing a Neural Network from Scratch in Python
#5Great, but the problem with these summaries is no real-world data/use-case. An actual business case example with code is easier to comprehend for less math-inclined folks.
Re: Implementing a Neural Network from Scratch in Python
#6how come every information like this, intended for beginners, don't say why you even need NN or when would you even use it.
Re: Implementing a Neural Network from Scratch in Python
#7Great, but the problem with these summaries is no real-world data/use-case. An actual business case example with code is easier to comprehend for less math-inclined folks.
i agree, especially for beginners.
Re: Implementing a Neural Network from Scratch in Python
#8Great, but the problem with these summaries is no real-world data/use-case. An actual business case example with code is easier to comprehend for less math-inclined folks.
[deleted]
Re: Implementing a Neural Network from Scratch in Python
#9There are so many little details to remember when you implement a Neural Network from "scratch". Or, I suppose, even if you do not. You know what would be a great contribution? An extensive set of unit tests, or even just problems with solutions. That way people can write their own implementations and test them. And even if a person were to implement the net in Pytorch of Tensorflow, they could test the work. So ther…
Built-in testing would be fantastic. Being able to tell if you designed a model wrong or just made an error setting up the code would reduce frustration a ton!
Re: Implementing a Neural Network from Scratch in Python
#10"We’ll use an optimization algorithm called stochastic gradient descent (SGD) that tells us how to change our weights and biases to minimize loss. It’s basically just this update equation:
w1 ← w1 − η ∂w|∂L
η is a constant called the learning rate that controls how fast we train. All we’re doing is subtracting η ∂w|∂L from w1:
- If ∂L | ∂w1 is positive, w1 will decrease, which makes L decrease
- If ∂L | ∂w1 is positive, w1 will increase, which makes L decrease"
Simple indeed.