Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Build a Neural Network

enlight.nyc

Post reply on HN
Print view View original

11–20 of 49 posts

Re: Build a Neural Network

#11

Post by HuShifang »

Anon84 wrote: post #7
fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

As someone who does teach tutorials as a side gig, I would argue that implementing matrix operations in a tutorial on neural networks is overkill. No matter what the level of the tutorial you always need to draw a line and assume a certain amount of background knowledge and knowing how to use standard tools isn't too much to ask. (yes, I know numpy isn't part of python's standard library, but it comes with pretty muc…

A new edition of Grus comes out next week actually...

https://www.amazon.com/Data-Science-Scratch-Principles-Pytho...

Re: Build a Neural Network

#12

Post by Anon84 »

HuShifang wrote: post #11
Anon84 wrote: post #7

Earlier quoted context omitted.

As someone who does teach tutorials as a side gig, I would argue that implementing matrix operations in a tutorial on neural networks is overkill. No matter what the level of the tutorial you always need to draw a line and assume a certain amount of background knowledge and knowing how to use standard tools isn't too much to ask. (yes, I know numpy isn't part of python's standard library, but it comes with pretty muc…

A new edition of Grus comes out next week actually... https://www.amazon.com/Data-Science-Scratch-Principles-Pytho...

Nice! He mentioned he was working on it when I met him at Strata last year, but I didn't know it was coming out already.

Re: Build a Neural Network

#15

Post by markbnj »

Seems like a good intro and I plan to work through it later. I've been learning a lot from Michael Nielsen's book, available at http://neuralnetworksanddeeplearning.com/index.html. He doesn't shy away from the underlying math, and his appreciation for it comes through in the writing. Even without a strong math background I was able to punch through the notation and figure things out.

Re: Build a Neural Network

#16

Post by jorgeleo »

This tutorial explained to me at the exact level of detail:

https://mattmazur.com/2015/03/17/a-step-by-step-backpropagat...

It was detailed enough for me to do all the calculations in an excel workbook, 1 complete cycle (forward, backward, and forward with the learned weights)

https://1drv.ms/x/s!Ar06sKFtc9d7goR5WQLo-RkB0XvWAA

Which allowed me to play with the name and factors to understand better how they impact the network as a whole.

Re: Build a Neural Network

#17

Post by asdfman123 »

fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

It is easy to do unless you don't code at all, or are completely confused by the math.

I'm a C# developer and I'm sure it would take me all of about 30 seconds to install a matrix multiplication package through nuget. I'm sure it would be immediately obvious how to add items to matrices or do a dot product.

Re: Build a Neural Network

#18

Post by asdfman123 »

gnulinux wrote: post #9
fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

Matrix math is easy peasy. Freshman level programming. Just lookup algorithms on Wikipedia and you're all set. The problem is it's extremely hard to make it efficient. Dozens of men-years are spent trying to optimize linear algebra libraries. There are handful linalg libraries that have competitive performance. It was my college project to make a fast linalg library, and boy it is fast. There are some things like mat…

Can anyone simply explain the gist of how matrix multiplication is optimized? I know a lot of is farmed out to the GPU (if you've got a good GPU), but what's the essence of it? Caching? Some kind of clever mathematical tricks? All of the above?

Re: Build a Neural Network

#19

Post by asdfman123 »

cwt137 wrote: post #13

If you think this blog article is lacking, get "Make Your Own Neural Network" by Tariq Rashid[1]. It is way more comprehensive, but still easy to comprehend. It also uses Python to create NN from scratch. 1. https://www.amazon.com/Make-Your-Own-Neural-Network/dp/15308...

Also, Andrew Ng's course on Coursera is free if you want to really learn it and have a few weeks to throw at it.

Re: Build a Neural Network

#20

Post by quantumOctopus »

HuShifang wrote: post #11
Anon84 wrote: post #7

Earlier quoted context omitted.

As someone who does teach tutorials as a side gig, I would argue that implementing matrix operations in a tutorial on neural networks is overkill. No matter what the level of the tutorial you always need to draw a line and assume a certain amount of background knowledge and knowing how to use standard tools isn't too much to ask. (yes, I know numpy isn't part of python's standard library, but it comes with pretty muc…

A new edition of Grus comes out next week actually... https://www.amazon.com/Data-Science-Scratch-Principles-Pytho...

Ugh I just bought the old one a week ago.
Post reply on HN