Live data from Hacker News

MNIST for ML Beginners: The Bayesian Way

alpha-i.co

21–30 of 37 posts

Re: MNIST for ML Beginners: The Bayesian Way

#21
post #3

I wish someone would come up with a tutorial for ML for the mathematically challenged. Something more practical and less theoretical.

Try "Neural Networks Demystified" by "Welch Labs" on youtube.

https://www.youtube.com/watch?v=bxe2T-V8XRs&list=PLiaHhY2iBX...

Re: MNIST for ML Beginners: The Bayesian Way

#22
Never use predicting stock prices as an example of anything. Pick literally anything other than stock prices to display confidence bounds. This helpful example diagram violates the EMH even worse by showing a large predictable directional change. You might as well illustrate physics with a chart of bowling balls falling upward.

Re: MNIST for ML Beginners: The Bayesian Way

#23
post #19
post #17

Earlier quoted context omitted.

This strategy is equally effective for most things in life.

> This strategy is equally effective for most things in life. Not if we understand 'effective' to also mean 'cost & time effective' You'd learn a lot by building a nuclear reactor from first principles, but it's not the most effective way to develop an intuition about how one operates.

I think you want to talk about whether the strategy is efficient, which I agree it is not. However, if you already tried understanding several general descriptions and it didn't work out, implementing something from scratch is an inefficient but effective way of really grokking it.

Re: MNIST for ML Beginners: The Bayesian Way

#24
post #3

I wish someone would come up with a tutorial for ML for the mathematically challenged. Something more practical and less theoretical.

So, there's a lot of material out there but it's disjointed. If you get annoyed implementing your own neural net with only Python + Numpy, you might try some more complex examples just to immediately point at them and say "I ran this" and "It did that". (The article uses neural networks so I'm addressing that here even though your question and the article title use the much broader 'ML'.)

1) Brandon Rohrer, now Data Scientist at Facebook, has a few great talks, including one on Bayes' Theorem/Bayesian Inference - https://m.youtube.com/playlist?list=PLVZqlMpoM6kbaeySxhdtgQP...

2) When asking future data scientists what tutorials for ML/NNs they like, they have usually found http://machinelearningmastery.com/ through Google and swear by it.

3) Josh Gordon, Developer Advocate at Google, has some simple ML/DL videos up in a 'Recipes' playlist: https://m.youtube.com/playlist?list=PLOU2XLYxmsIIuiBfYad6rFY...

If you want to just step through other people's code, you can do that too. Disclaimer: I put the below list together and it's not for ML broadly but for DL. That said if you want to run some examples fast and see the output, a number of folks have made that work for you -

I for one was floored to find great iOS examples (admittedly now deprecated for iOS 11). But If you have an iPhone with Metal (5s and up) Matthijs Hollemans - who wrote the iOS Apprentice at Ray Wenderlich - has Inception, YOLO, and MobileNets pre-trained and ready to go using Xcode, and it's fun to watch them work on your phone - https://medium.com/@SamPutnam/deep-learning-download-and-run...

Re: MNIST for ML Beginners: The Bayesian Way

#25
post #3

I wish someone would come up with a tutorial for ML for the mathematically challenged. Something more practical and less theoretical.

ML really requires more than a tutorial. Just bite the bullet and take Andrew Ng's ML class.

To understand ML you've got to have at least a basic understanding of the math. And it's really not that difficult, especially if you find the right class/book/professor/etc.

The problem is that there are a ton of terrible writers and instructors out there.

I think it's just as important to ignore the terrible stuff (pretty much any blog post on ML) as it is to learn from the good stuff (e.g. Ng's ML course).

Re: MNIST for ML Beginners: The Bayesian Way

#27
post #22

Never use predicting stock prices as an example of anything. Pick literally anything other than stock prices to display confidence bounds. This helpful example diagram violates the EMH even worse by showing a large predictable directional change. You might as well illustrate physics with a chart of bowling balls falling upward.

Could you elaborate why predicting stock prices is an invalid example?

Re: MNIST for ML Beginners: The Bayesian Way

#28
post #15
post #8

Earlier quoted context omitted.

Honestly, the best thing you can do is try to implement your own shitty neural net with only Python + Numpy, from scratch, with only a basic understanding of the math. It will make most of the math very concrete very fast.

Naive question : is this strategy equally effective for all ML models?

Some require a lot more understanding than others (for instance I'm not sure I'd be comfortable implementing a kernelized SVM from scratch, even though intuitively I know how it works) but basic neural networks (simple perceptron, simple feedforward network, simple recurrent network) are quite easy to grasp, and backpropagation is very intuitive. You can even use finite difference approximation [1] to bypass the derivatives when you're starting (at the cost of some efficiency) and figure out the rest as you go.

[1] https://en.wikipedia.org/wiki/Finite_difference

Re: MNIST for ML Beginners: The Bayesian Way

#29
post #9
post #8

Earlier quoted context omitted.

Honestly, the best thing you can do is try to implement your own shitty neural net with only Python + Numpy, from scratch, with only a basic understanding of the math. It will make most of the math very concrete very fast.

Cannot upvote this enough. I think this is the only way to really grok backpropagation. The hours of staring at the update formula till your eyes glaze over the subscripts and superscripts and the summations would not give you as good an understanding as implementing a toy neural net with just a single hidden layer. Its actually a whole lot easier than parsing those low-level notation. It can be done better with high…

Yes, once you've dealt with implementing it, the notation just makes sense as the most compact way of formalizing what you just did.

Re: MNIST for ML Beginners: The Bayesian Way

#30
post #13
post #8

Earlier quoted context omitted.

Honestly, the best thing you can do is try to implement your own shitty neural net with only Python + Numpy, from scratch, with only a basic understanding of the math. It will make most of the math very concrete very fast.

Good idea! I'll give it a go

Don't bother. The guy that wrote Encog has several yt tutorials on how he did it and there are hundreds of others.

I learned nothing from impl my own other than why all NN libs break when you input values > 1.

Any decent NN lib will be way better that whatever you could write in a week full time.

Pick your fav lang, find the most used NN lib and try a kaggle competition.

If you can get to about rank 50% your training is complete.

Post reply on HN