Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MNIST Handwritten Digit Classifier – beginner neural network project

github.com

Post reply on HN
Print view View original

11–20 of 31 posts

Re: MNIST Handwritten Digit Classifier – beginner neural network project

#11

Post by ironislands »

imurray wrote: post #3

Nice. Recreating these methods in simple code for yourself is definitely the way to check you understand it. This demo looks nice, clean, and straightforward. (Although I'd rename or comment variables x and y, or give some sort of guidance on what way around the weight matrices are within the code itself.) It's also worth checking out existing neural net code-bases to see what tricks they have. The fine details usual…

Hi ! Thanks for the valuable suggestion :-) Your points make much sense to me. I am caught up with some other work but I surely intend to make amendments latest by next 20 days.

Re: MNIST Handwritten Digit Classifier – beginner neural network project

#12

Post by netvarun »

tyrael71 wrote: post #4

Can someone explain how this repo is so popular/ why it's so popular here? This is a basic implementation of a relatively simple algorithm that you learn when initially starting with ML/DL. I do not want in any way to sound critical and am genuinely curious about the dynamics of why people would find this interesting given it's reduced complexity.

I up-voted this since I found it really helpful while working through Michael Nielsen's book on neural networks[1]. (I had chanced upon this repo a couple of months ago)

The code is actually based on the original code from the book (e.g.: can be seen from the variable names like 'nabla') , but written in a more succinct manner. Since I am relatively new to Python, I found it easier to follow this repo's code than the code in the book and used it as my reference implementation.

It's missing quite a few things like calculating accuracy, regularization, etc. but they are quite straightforward to implement.

[1] Neural Networks and Deep Learning by Michael Nielsen - http://neuralnetworksanddeeplearning.com

Re: MNIST Handwritten Digit Classifier – beginner neural network project

#13

Post by imurray »

tyrael71 wrote: post #8
imurray wrote: post #3

Nice. Recreating these methods in simple code for yourself is definitely the way to check you understand it. This demo looks nice, clean, and straightforward. (Although I'd rename or comment variables x and y, or give some sort of guidance on what way around the weight matrices are within the code itself.) It's also worth checking out existing neural net code-bases to see what tricks they have. The fine details usual…

'It's also worth checking out existing neural net code-bases to see what tricks they have. The fine details usually aren't in papers, and they're not all in the text-books either.' Given that you are a person who is highly-qualified to answer, I am genuinely curious why do you think that is? Reimplementing algorithms from scratch is an efficient way to learn, understand the underlying concepts and attempt improvement…

A lot of machine-learning papers are eight pages. Speech conference papers (heavy users of neural nets) are often only four. Some details aren't part of the main message, so don't make it in. Often code is available, and initialization and other tweaks can be found in there (even if you aren't going to use their code).

That said, there are also whole papers, even collected volumes, on initialization and other practical details.

Textbooks aren't always up-to-date with the latest practical knowledge, as deep-learning practice is moving quickly. Or they simply don't want to clutter their high-level maths descriptions with code-level implementation details. Teaching stuff is all about tradeoffs. I'm sure several books do mention the scale of weights for simple feed-forward weights though, as it's not an implementation-level detail, and it's probably been well known since the 1980s.

Re: MNIST Handwritten Digit Classifier – beginner neural network project

#14

Post by superuser2 »

tyrael71 wrote: post #4

Can someone explain how this repo is so popular/ why it's so popular here? This is a basic implementation of a relatively simple algorithm that you learn when initially starting with ML/DL. I do not want in any way to sound critical and am genuinely curious about the dynamics of why people would find this interesting given it's reduced complexity.

We aren't all working on machine learning. There will always be some people who are getting into it for the first time.

Re: MNIST Handwritten Digit Classifier – beginner neural network project

#15

Post by rch »

tyrael71 wrote: post #4

Can someone explain how this repo is so popular/ why it's so popular here? This is a basic implementation of a relatively simple algorithm that you learn when initially starting with ML/DL. I do not want in any way to sound critical and am genuinely curious about the dynamics of why people would find this interesting given it's reduced complexity.

https://xkcd.com/1053/

Re: MNIST Handwritten Digit Classifier – beginner neural network project

#16

Post by d4rth_s1d10us »

rch wrote: post #15
tyrael71 wrote: post #4

Can someone explain how this repo is so popular/ why it's so popular here? This is a basic implementation of a relatively simple algorithm that you learn when initially starting with ML/DL. I do not want in any way to sound critical and am genuinely curious about the dynamics of why people would find this interesting given it's reduced complexity.

https://xkcd.com/1053/

Haha nice comic ! :-D

Re: MNIST Handwritten Digit Classifier – beginner neural network project

#18

Post by RockyMcNuts »

badminton1 wrote: post #17

If you want to jump into this sort of thing, I highly recommend Google's Udacity course for deep learning. https://classroom.udacity.com/courses/ud730

it's very good, but personally I'd start with the Andrew Ng and Hinton courses

https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/neural-networks

I think the Udacity course is best if you know principles of machine learning and want to apply them in a more professional toolchain and learn Tensorflow

Re: MNIST Handwritten Digit Classifier – beginner neural network project

#19

Post by [deleted] »

RockyMcNuts wrote: post #18
badminton1 wrote: post #17

If you want to jump into this sort of thing, I highly recommend Google's Udacity course for deep learning. https://classroom.udacity.com/courses/ud730

it's very good, but personally I'd start with the Andrew Ng and Hinton courses https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/neural-networks I think the Udacity course is best if you know principles of machine learning and want to apply them in a more professional toolchain and learn Tensorflow

[deleted]

Re: MNIST Handwritten Digit Classifier – beginner neural network project

#20

Post by sapphireblue »

I have an old one written some time ago by myself too, in node.js/javascript: https://github.com/crystalline/dnnjs It is a simple multilayer perceptron with ReLU nonlinearity, it can achieve 1.7% error on MNIST which is bad compared to convnets but good enough for multilayer perceptron. Training a model is as simple as running "node node-mnist.js" in terminal.
Post reply on HN