Neural Networks for Machine Learning • Hacker News Board

Re: Neural Networks for Machine Learning

#21

Post by waterlesscloud » Wed, Oct 03, 2012, 3:55 AM UTC

FaceKicker wrote: post #18

lathamcity wrote: post #8

I'm in the middle of the machine learning coursera course, and registered for this one as well due to interest in the material. My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was…

> My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was just a syntax challenge, and that got irritating after a bit so I stopped doing them. I agree with this. The programming assig…

I'm doing the Computational Finance class via Coursera at the moment, and I've done a number of other courses previously.

I agree the programming assignments in the Finance class tend to be too simple. Most of the code is literally handed to you, you just have to understand it well enough to change it. I also understand that even that can be a major challenge if you don't have the background for it.

But I'm choosing to see the class itself as a starting point. It's a framework for my own explorations into the topics. I can do the minimum and get the minimum out of it. Or I can use what's provided as a base and go further.

The Coursera Algorithms class, for example. Writing code that got the answer was relatively easy, so once that step was done it became about optimizing the code for my own learning benefit.

It's like any educational process, you get our what you put in.

Re: Neural Networks for Machine Learning

#22

Post by MaysonL » Wed, Oct 03, 2012, 3:56 AM UTC

lathamcity wrote: post #8

I'm in the middle of the machine learning coursera course, and registered for this one as well due to interest in the material. My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was…

I took the course in the spring, and found it interestng, and the programmin assignments fairly easy. This summer I took the ML course that Caltech offered, which was significantly more challenging (the homework assignments were multiple choice, but they often required writing substantial code, without any starter code.) The Caltech course is now available on iTunes U...

Re: Neural Networks for Machine Learning

#23

Post by azakai » Wed, Oct 03, 2012, 3:59 AM UTC

> Neural Networks are gradually taking over from simpler Machine Learning methods

And haven't SVMs and such gradually taken over from Neural Networks?

Re: Neural Networks for Machine Learning

#24

Post by jackpirate » Wed, Oct 03, 2012, 4:03 AM UTC

lathamcity wrote: post #8

I'm in the middle of the machine learning coursera course, and registered for this one as well due to interest in the material. My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was…

I'm positive they wouldn't complain about not getting to retake quizzes after getting poor grades.

My experience is that students everywhere complain about grading. I've never been to Stanford, but I've attended and worked at several other top tier universities.

Re: Neural Networks for Machine Learning

#25

Post by whitewhim » Wed, Oct 03, 2012, 4:16 AM UTC

FaceKicker wrote: post #18

lathamcity wrote: post #8

I'm in the middle of the machine learning coursera course, and registered for this one as well due to interest in the material. My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was…

> My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was just a syntax challenge, and that got irritating after a bit so I stopped doing them. I agree with this. The programming assig…

I've noticed this to be the case too with other courses. So for this one I've decided to implement everything in Scala (I'm currently taking the functional programming course as well. This will work well since this machine learning course requires no code submission and just questions about the results

Re: Neural Networks for Machine Learning

#26

Post by vitno » Wed, Oct 03, 2012, 4:18 AM UTC

rubashov wrote: post #11

I tried to do a couple coursera courses and found the video lectures highly inefficient; very needlessly time consuming, even watching them sped up. All I really want is a glorified text book with quiz grading and a final.

this. No offense to these professors, but what are they presenting in their video lectures that I can't garner from their writing?

Re: Neural Networks for Machine Learning

#27

Post by FaceKicker » Wed, Oct 03, 2012, 4:18 AM UTC

waterlesscloud wrote: post #21

FaceKicker wrote: post #18

Earlier quoted context omitted.

> My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was just a syntax challenge, and that got irritating after a bit so I stopped doing them. I agree with this. The programming assig…

I'm doing the Computational Finance class via Coursera at the moment, and I've done a number of other courses previously. I agree the programming assignments in the Finance class tend to be too simple. Most of the code is literally handed to you, you just have to understand it well enough to change it. I also understand that even that can be a major challenge if you don't have the background for it. But I'm choosing…

Right, you can get more out of the assignments if you try, but to me the purpose of assignments (versus passive learning - lectures, reading, etc.) is to force your brain to synthesize rather than just comprehend. The ideal assignment, then, is one that forces you to synthesize as many concepts it intends to teach as possible.

Just like you could go back and implement for yourself the skeleton code they handed you, you could also go out and implement everything in the lectures without any assignments at all. It's just that, like you said, the assignments provide a useful starting point. And I'm only saying they could be even more useful by requiring you to implement more of the complete pipeline.

The fact that an incredibly self-motivated person could learn everything there is to know about machine learning with the course as a starting point doesn't mean that it's bad to make the course more useful for a somewhat lazier or less interested person.

Re: Neural Networks for Machine Learning

#28

Post by xianshou » Wed, Oct 03, 2012, 4:23 AM UTC

lathamcity wrote: post #8

I'm in the middle of the machine learning coursera course, and registered for this one as well due to interest in the material. My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was…

I've taken both, and the code is in fact not that much simpler than it was in the original class. There are, however, two huge differences: the algorithm is spoon-fed to you, and there is no math.

Firstly, think about how much more difficult the assignments would be if, for example, the steps weren't broken out and we didn't get any advice on how to vectorize. Of course, it would still be short work for anyone who (a) knows Matlab/Octave and/or (b) understands the material well, but it would also be an order of magnitude harder.

Secondly - and this is by far the larger point - the original CS 229 was really about math; the programming assignments were more of an afterthought. The lectures and homework mainly focused on the theoretical derivations and corollaries of the math that led to the algorithms. Once you'd done your bit on the math and cried to your classmates and the TA about it, you could go and implement the beautiful and extremely succinct result in Matlab.

As for my perspective on the difference, I believe it is a deliberate choice made with full knowledge of the difficulty drop. For starters, there are (with regards to homework help) no TAs in this course, so the absolute difficulty would have to decline to create an equivalent experience. More significantly, the enrollment has increased by a factor of about 700. If Stanford students had trouble with the original, you can bet that the median student in the course doesn't find it as easy as either of us does. If the goal is to generate the greatest benefit for the most people, and delivering the algorithms with a good intuition on their proper use will do so, then this course has succeeded marvelously. Of course, the smartest and most dedicated students will want more, which remains available through textbooks as well as the original course handouts (http://cs229.stanford.edu/materials.html). However, I would argue that the goal of most MOOCs (massive open online courses) should be to kindle interest and foster basic understanding, both of which the Coursera version achieves.

Re: Neural Networks for Machine Learning

#29

Post by platz » Wed, Oct 03, 2012, 4:23 AM UTC

dvdhsu wrote: post #15

lathamcity wrote: post #8

I'm in the middle of the machine learning coursera course, and registered for this one as well due to interest in the material. My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was…

> The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. Right; I agree. I'm not sure how they would go about making it more challenging though. They can't expect us to go out and collect data ourselves, after all. I suppose they could give us the data, then expect us to code the setup and algorithms up ourselves, but that, too, would become r…

(PCM) -> Do you mean PCA (Pricipal Component Analysis)?

Re: Neural Networks for Machine Learning

#30

Post by nphrk » Wed, Oct 03, 2012, 4:25 AM UTC

azakai wrote: post #23

> Neural Networks are gradually taking over from simpler Machine Learning methods And haven't SVMs and such gradually taken over from Neural Networks?

Well not quite. While SVMs gained a lot of popularity for having nice properties e.g.

  1) a convex problem which means a unique solution and a lot of already existing technology can be used
  2) the "kernel trick" which enables us to learn in complicated spaces without computing the transformations
  3) can be trained online, which makes them great for huge datasets (here the point 2) might not apply - but there exist ways - if someone's interested I can point out some papers)

There is an ongoing craze about deep belief networks developed by Hinton (who is teaching this course) who came up with an algorithm that can train them (there exist local optima and such, so it's far from ideal). Some of the reasons they're popular

  1) They seem to be winning algorithm for many competitions / datasets, ranging from classification in computer vision to speech recognition and if I'm not mistaken even parsing. They are for example used in the newer Androids.
  2) They can be used in an unsupervised mode to _automatically_ learn different representations (features) of the data, which can be then used in subsequent stages of the classification pipeline. This makes them very interesting because while labelled data might be hard to get by, we have a lot of unlaballed datasets thanks to the Internet. As what they can do - see the work by Andrew Ng when they automatically learned a cat detector.
 3) They're "similar" to biological neural networks, so one might think they have the necessary richness for many interesting AI applications.