Build a Neural Network
enlight.nyc
Build a Neural Network
1–10 of 49 posts
Re: Build a Neural Network
#2As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers.
It's a legitimately valid part of machine learning, and its not easy to do for novices.
And I need help putting it on my badger damn it!
Re: Build a Neural Network
#3Here's another Neural Network from scratch that I found useful:
Re: Build a Neural Network
#4As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…
If you don't care much about performance (and if you are reimplementing a neural network from scratch you're probably doing it more as a learning project than anything else) implementing matrix operations isn't very difficult.
If you're not used to work with matrices simply reading the Wikipedia article might tell you enough to implement them yourself.
Re: Build a Neural Network
#5Would be great if this included real world data or application to understand context.
Re: Build a Neural Network
#6As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…
The complicated parts of Numpy are themselves a wrapper for the seminal LAPACK: http://www.netlib.org/lapack/. It has C language APIs, so that might help you with what you need to do.
Re: Build a Neural Network
#7As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…
As someone who does teach tutorials as a side gig, I would argue that implementing matrix operations in a tutorial on neural networks is overkill. No matter what the level of the tutorial you always need to draw a line and assume a certain amount of background knowledge and knowing how to use standard tools isn't too much to ask. (yes, I know numpy isn't part of python's standard library, but it comes with pretty much any Python distribution as many other libraries depend on it.)
If we're talking about a longer format, such as a book, then we might consider digging deeper and implementing as much as possible using the barest of Python requirements. Indeed, Joel Grus does implement everything from scratch in his great (although a bit dated) book https://www.amazon.com/Data-Science-Scratch-Principles-Pytho....
EDIT: This is still a work in progress (and relies on numpy and matplotlib), but here is my version: https://github.com/DataForScience/DeepLearning These notebooks are meant as support for a webinar so they might not be the clearest as standalone, but you also have the slides there.
Re: Build a Neural Network
#8As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…
I'm curious if you have a particular language/microcontroller/microwave/badger you have in mind? Depending on which, YMMV.
Re: Build a Neural Network
#9As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…
Matrix math is easy peasy. Freshman level programming. Just lookup algorithms on Wikipedia and you're all set.
The problem is it's extremely hard to make it efficient. Dozens of men-years are spent trying to optimize linear algebra libraries. There are handful linalg libraries that have competitive performance. It was my college project to make a fast linalg library, and boy it is fast. There are some things like matrix multiplication that if you implement in C with the trivial algorithm, takes >2 mins but with some tricks you can make it as fast as So, if you want to implement linalg in some language and compile it, go ahead, more power to you. But it's basically impossible to do it efficiently. My opinion is: this is fine and we should do this. There should be linalg libraries written in pure python (and are 1000x slower than lapack) but just understand that it's impossible to satisfy all use cases of numpy this way (at least currently).
Re: Build a Neural Network
#10As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…
If you don't care much about performance (and if you are reimplementing a neural network from scratch you're probably doing it more as a learning project than anything else) implementing matrix operations isn't very difficult. If you're not used to work with matrices simply reading the Wikipedia article might tell you enough to implement them yourself.
If you have an assembler or C compiler you can implement matrix multiplication (GEMM) which usually does most of the heavy lifting in your neural net. Now you correctly alluded that it may not be simple to efficiently implement GEMM but if you have a simple architecture without a complex memory hierarchy then using whatever SIMD facilities are available and some standard tricks will get you in ballpark of peak FLOP/s.
Or, just download a fast BLAS from your hardware vendor...