Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning Machine Learning: A beginner's journey

muratbuffalo.blogspot.com

Post reply on HN
Print view View original

21–30 of 56 posts

Re: Learning Machine Learning: A beginner's journey

#21

Post by placebo »

saurabhjha wrote: post #10

I think this "machine learning for hackers" approach is just not enough. Oftentimes, you do need a solid theoretical/mathematical background. Most people seems to approach ML like they approach programming tools or libraries - learn just enough to get job done and move on. I was studying machine learning from Andrew Ng's CS229 (the class videos are online. I think they date from 2008 or hereabout). There is no way yo…

I understand where you're coming from and also agree in principle, but I'd change the claim that "this approach is just not enough" with "this approach is just not enough for achieving many things in machine learning including breaking new ground". I think there's always a way to be creative within the constraints and concepts/axioms you take as given. For example, the fact that I have absolutely no control (or knowledge for that matter) on how to design or improve the microprocessor on my computer, I don't feel this is limiting my creativity in software development at all. Once changes and improvements occur in the hardware level, I'm sure they will find a way to the software development layer and then I'll be handed even more degrees of freedom to be creative (though I'm not complaining with the freedom I currently have). Don't you think the same might apply to machine learning - i.e those with solid theoretical/mathematical background are analogous to the chip designers and the "machine language hackers" are the software developers?

Re: Learning Machine Learning: A beginner's journey

#22

Post by Philipp__ »

Distributed Systems and ML are probably two most interesting things that I have on the radar, that got me really scared to the point where I do not know from where to start, and most importantly for what?! Most of my free time (time I spent on personal projects) was writing physics simulation in Java, playing with Lisp and doing some backend development. Nothing amazing. Year and a half ago I got really interested into Operating systems (tried FreeBSD and blew my mind) and played with Docker. And at the end of this year, I am like: "Ok Philip what shall I focus on for year to come?" And the thing is If I choose to go Ai route, I do not know from where to start (I consider my math background to be pretty good, I was studying EE before I dropped out after 2 years, and enrolled to CS, done all of the math courses which were pretty rough), Ai/ML looks interesting but it looks so high level to program and so abstract to understand. It's really looking like arcane magic to me. With Dist. Systems is that I have a feeling that is more "engineering" and "industrial" thing, where you can't do much by yourself at home, besides reading and writing some code in relevant languages about backend, sometimes lower level, and learning about systems and computer innards. And the third option was to go and play with Erlang/Elixir, which is most attractive since results will come pretty soon, and may be relevant form my interest in Distributed systems.

Re: Learning Machine Learning: A beginner's journey

#23

Post by devoply »

thw1254 wrote: post #20
saurabhjha wrote: post #10

I think this "machine learning for hackers" approach is just not enough. Oftentimes, you do need a solid theoretical/mathematical background. Most people seems to approach ML like they approach programming tools or libraries - learn just enough to get job done and move on. I was studying machine learning from Andrew Ng's CS229 (the class videos are online. I think they date from 2008 or hereabout). There is no way yo…

I've been interviwing for ML positions and what struck me was the general disdain for details. I had one manager claim that they're set to beat their competitors now because they're moving to GOOG's new tensorflow. Others knew little more than Tensoflow and Backprop. Frankly I regret spending time understanding all the math, instead of working for some company, munging through their data and applying some black box s…

Yeah, but is it web scale?

Re: Learning Machine Learning: A beginner's journey

#24

Post by chestervonwinch »

Philipp__ wrote: post #22

Distributed Systems and ML are probably two most interesting things that I have on the radar, that got me really scared to the point where I do not know from where to start, and most importantly for what?! Most of my free time (time I spent on personal projects) was writing physics simulation in Java, playing with Lisp and doing some backend development. Nothing amazing. Year and a half ago I got really interested in…

> If I choose to go Ai route, I do not know from where to start ...

This type of comment is often made in machine learning (ML) related submissions.

The pre-req list is long: calculus, linear algebra, stats, probability, numerical methods (for optimization, linear algebra, maybe interpolation), etc. BUT, you don't really need to go through the entirety of each subject for ML. For example, in calculus, you probably only need to focus on the aspects necessary for optimization, rather than integral techniques, convergence of sequences, etc. The trouble is that it is difficult to know which subtopics of each subject are worth spending time on unless you already know machine learning (or you have the luxury of someone with experience guiding you).

The latter difficulty is compounded by the fact that there seems to be many more resources (at least posted as popular submission on the web) for learning neural nets or learning some specific framework to implement neural networks, than to learn the mathematical and statistical foundations of ML. This is fine -- neural nets are a popular and powerful model, and people like to work on something tangible to get acquainted with a topic.

I wonder if people might enjoy a well-written textbook covering the basic math for ML -- something like, "All the math you missed (but need to know for machine learning)" [1]. I might enjoy working on such an ebook if there was desire for one, but my time is pretty limited (like most).

[1]: https://www.amazon.com/All-Mathematics-You-Missed-Graduate/d...

Re: Learning Machine Learning: A beginner's journey

#25

Post by Eridrus »

epberry wrote: post #15
Dzugaru wrote: post #6

Earlier quoted context omitted.

Pretty much this. In the real world you have to deal with - Too few labeled / garbage labeled data (70000 digits? How about only 1000 complex class objects?) - Obscure bugs in custom implementation (yeah my custom layer works and gradient is correct... or wait why it diverges after 10k iteration? hmm). - Timing/RAM constraints (it should segment an image under 10ms on Jetson TX1, well, good luck with GoogLeNet)

Nailed it. Neural net execution speed is so critical for may production systems and it's very difficult to hit the sweet spot on trade offs but I never hear about these issues in wild.

> Nailed it. Neural net execution speed is so critical for may production systems and it's very difficult to hit the sweet spot on trade offs but I never hear about these issues in wild.

That's probably because you're not listening. There's plenty of literature on scaling down neural nets to smaller devices because everyone knows it's an issue and can trivially get a smaller device, see techniques such as Quantization , Distillation, Pruning or architectures specifically designed for the task such as YOLO.

Re: Learning Machine Learning: A beginner's journey

#26

Post by forgetsusername »

saurabhjha wrote: post #10

I think this "machine learning for hackers" approach is just not enough. Oftentimes, you do need a solid theoretical/mathematical background. Most people seems to approach ML like they approach programming tools or libraries - learn just enough to get job done and move on. I was studying machine learning from Andrew Ng's CS229 (the class videos are online. I think they date from 2008 or hereabout). There is no way yo…

>Well, I think a lot of competitive advantage of Google/Facebook

But 99.9% of people are not trying to compete with Google or Facebook. They are looking for basic insights.

I do agree that ML isn't and shouldn't be "easy". But we inevitably and thankfully get abstracted away from the inner workings of things over time.

Re: Learning Machine Learning: A beginner's journey

#27

Post by pramodliv1 »

chestervonwinch wrote: post #24
Philipp__ wrote: post #22

Distributed Systems and ML are probably two most interesting things that I have on the radar, that got me really scared to the point where I do not know from where to start, and most importantly for what?! Most of my free time (time I spent on personal projects) was writing physics simulation in Java, playing with Lisp and doing some backend development. Nothing amazing. Year and a half ago I got really interested in…

> If I choose to go Ai route, I do not know from where to start ... This type of comment is often made in machine learning (ML) related submissions. The pre-req list is long: calculus, linear algebra, stats, probability, numerical methods (for optimization, linear algebra, maybe interpolation), etc. BUT, you don't really need to go through the entirety of each subject for ML. For example, in calculus, you probably on…

Metacademy [1] does a good job of identifying which subtopics of each subject are relevant to ML.

[1] https://metacademy.org/roadmaps/

Re: Learning Machine Learning: A beginner's journey

#28

Post by pramodliv1 »

ken47 wrote: post #12
saurabhjha wrote: post #10

I think this "machine learning for hackers" approach is just not enough. Oftentimes, you do need a solid theoretical/mathematical background. Most people seems to approach ML like they approach programming tools or libraries - learn just enough to get job done and move on. I was studying machine learning from Andrew Ng's CS229 (the class videos are online. I think they date from 2008 or hereabout). There is no way yo…

I don't think Andrew Ng would agree with your assertion. His Coursera ML class assumes little more than a basic high school math education, and at the start of the course, he teaches the very small subset of linear algebra required to understand his course materials. I think what Andrew Ng would say is that without a rigorous statistical background, you will be limited in your ability to use ML, and you will certainl…

> And at the start of the course, he teaches the very small subset of linear algebra required to understand his course materials.

I tried doing the ML course without any prior knowledge of Linear Algebra and dropped out after the first three weeks. In hindsight, I realized it wouldn't have been possible to appreciate how PCA works without understanding eigenvectors, how collaborative filtering is an elegant application of matrix factorization and so on.

But after I completed Strang's Linear Algebra course, the entire ML class was a breeze.

Re: Learning Machine Learning: A beginner's journey

#29

Post by bko »

saurabhjha wrote: post #10

I think this "machine learning for hackers" approach is just not enough. Oftentimes, you do need a solid theoretical/mathematical background. Most people seems to approach ML like they approach programming tools or libraries - learn just enough to get job done and move on. I was studying machine learning from Andrew Ng's CS229 (the class videos are online. I think they date from 2008 or hereabout). There is no way yo…

An alternate theory is that you can make use of ML to do useful tasks through an understanding of just high school math (basic algebra) and basics of python. I'm not sure if that's actually true, but I'm inclined to be in this camp as abstractions are used in virtually every other tasks. The amount of extra value that can be had from applying even basic ML techniques is so great, that there is probably a lot of upside to using ML even if they're only able to hire practitioners.

A great resource specifically tailored to those that don't have a especially strong grasp of probability and statistics is Grokking Deep Learning

https://iamtrask.github.io/2016/08/17/grokking-deep-learning...

Re: Learning Machine Learning: A beginner's journey

#30

Post by Philipp__ »

chestervonwinch wrote: post #24
Philipp__ wrote: post #22

Distributed Systems and ML are probably two most interesting things that I have on the radar, that got me really scared to the point where I do not know from where to start, and most importantly for what?! Most of my free time (time I spent on personal projects) was writing physics simulation in Java, playing with Lisp and doing some backend development. Nothing amazing. Year and a half ago I got really interested in…

> If I choose to go Ai route, I do not know from where to start ... This type of comment is often made in machine learning (ML) related submissions. The pre-req list is long: calculus, linear algebra, stats, probability, numerical methods (for optimization, linear algebra, maybe interpolation), etc. BUT, you don't really need to go through the entirety of each subject for ML. For example, in calculus, you probably on…

Thanks, that was the answer I was looking for, you said it much better than I did! When i look at Ai/ML I see a lot of mathematics, not frameworks and programming languages. Anyone can learn to use specific framework or adopt to certain programming language and environment. What concerned me was mathematics wise, since on EE course Math was much more apply oriented, with integration techniques and geometry, not so much about statistics and probability.
Post reply on HN