This is very cool and informative. Thank you for this article!
Implementing a Neural Network from Scratch in Python
101–104 of 104 posts
Re: Implementing a Neural Network from Scratch in Python
#102Earlier quoted context omitted.
>> People who are reading “math” code can be expected to understand mathematical notation, e.g. to be capable of reading a journal paper where an algorithm or formula is described more completely including motivation, formal derivation, proofs of correctness, proofs of various formal properties, ... The majority of machine learning papers are very well stocked in terms of heavy mathematical-y notation, but are very,…
If you're working in a domain, is it really that much to ask to become familiar with it? Especially if the domain has a large theoretical component. When we teach people software engineering we teach them concepts like "give your variables meaningful names". Now that we're in sub-domain of implementing some mathematics in software, I'd argue that matching the variables and functions to their source (more or less) _is…
>> If you're working in a domain, is it really that much to ask to become familiar with it? Especially if the domain has a large theoretical component.
If the domain has a large theoretical component. Here, we're talking about statistical machine learning and neural networks in particular, where this is, for the most part, not at all the case.
>> And you would immediately lose the ability to quickly and easily recognise similar patterns and abstractions that mathematical notation so fluently allows.
I disagree. An algorithm is mathematical notation, complete with immediately recognisalbe patterns and abstractions (for-loops, conditional blocks, etc).
And, btw, so is computer code: it is formal notation that, contrary to mathematical formulate that require some familiarity with the conventions of a field to read and understand, has an objective interpretation- in the form of a compiler for the language used.
So machine learning papers could very well notate their algorithm in Python, even a high-level abstraction (without all the boilerplate) of the algorithm, and that would actually make them much more accessible to a larger number of people.
Mathematical notation, as in formulae, is not required- it's a tradition in the field, but that's all.
However, that's a bit of a digression from the subject of the naming of variables. Apologies. It's still relevant to the compreshensibility of formal notation.
Re: Implementing a Neural Network from Scratch in Python
#103Earlier quoted context omitted.
For what it's worth, I didn't read the article at all, went straight to the final code, and had no issue reading or understanding it. The variable names seem fine to me. They're actually quite good, and I liked the convention for naming partial derivatives. This code is designed to be read by someone who is pretty familiar with the underlying concepts. If you don't really know how a neural network works or how gradie…
Well then they should get rid of the misleading title “from scratch” it makes it sound like they are going to explain it without beltching out a bunch of abstract math. I’m really annoyed that everyone pretends that you need math to explain a NN. I have done plenty of them without numpy or math using just arrays and loops.
This is not abstract math, and the article does explain what it's doing before presenting the code snippets.
How can you explain or implement gradient descent without math? At some point I think you have to accept that this is a topic that involves math, and you're way better off understanding it on those terms rather than trying to avoid it.
Re: Implementing a Neural Network from Scratch in Python
#104Great, but the problem with these summaries is no real-world data/use-case. An actual business case example with code is easier to comprehend for less math-inclined folks.
The problem with that is these kinds of simplified "how it works" neural net examples don't scale for real world problems - especially when using an interpreted language like Python. Even a simple but useful scenario like the MNIST number recognition system would probably run "dog-slow" on such a neural network, given the number of input nodes, plus the size of the hidden layer - the combinatorial "explosion" of edge…
Maybe you missed the disclaimer ?
"The code below is intended to be simple and educational, NOT optimal."