Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

spectrum.ieee.org

Post reply on HN
Print view View original

51–60 of 60 posts

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#51

Post by muraiki »

michaelochurch wrote: post #42
muraiki wrote: post #38

Earlier quoted context omitted.

As someone who has tried out various MOOCs and entry level resources on machine learning, this is the same conclusion I came to. Beyond any sort of trivial example, I found I lacked the mathematical and statistical knowledge to not only interpret the results in a relatively unbiased and error-free way, but to know "what to do next." What scares me is that MOOCs are really pushing the data scientist field -- see Udaci…

Beyond any sort of trivial example, I found I lacked the mathematical and statistical knowledge to not only interpret the results in a relatively unbiased and error-free way, but to know "what to do next." The popular MOOCs don't take you far enough to start doing serious machine learning, but you don't need a PhD to be ready to solve those problems. It takes work. Lots of work. Re-learn linear algebra until you know…

Thank you for your advice. In my case it's not "re-learn linear algebra" but "learn linear algebra... after first learning calculus and how to understand/write a proof." :) At 32 I'm not certain if this is a worthwhile way for me to go...

That being said, I haven't given up completely. I'm starting to read "The Haskell Road to Logic, Maths, and Programming" in the hopes of finally being able to grok proofs. At the very least, I feel that learning more math can only help me as a developer.

For others reading this, this edx course on Probability seemed like it was really good, until my lack of maths background caught up to me: https://www.edx.org/course/mitx/mitx-6-041x-introduction-pro... For Linear Algebra, check out http://www.ulaff.net/

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#52

Post by yid »

blauwbilgorgel wrote: post #39
Daishiman wrote: post #24

Great interview. In my experience it's amazing just how many people are talking about "Big Data" and just how exactly none of those are the ones with the necessary PhDs in statistics and algorithms to get anything of any value done. In my experience there are very few domains within Machine Learning where you don't need to be an expert in the field to yield useful conclusions out of the data. Even if you have a high-…

> ... how exactly none of those are the ones with the necessary PhDs in statistics and algorithms to get anything of any value done. I see it almost the other way around: Companies strictly demand PhD's for Big Data jobs and can't find this unicorn. Yet we live in a time where we don't need a PhD program to receive education from the likes of Ng, LeCun and Langford. We live in a time where curiosity and dedication ca…

While there's some truth in what you're saying, you sort of demonstrate a very common pitfall:

> Tuning parameters is basically a gridsearch. You can bruteforce this. In goes some ranges of parameters, out come the best params found.

This sounds so simple. However, if you just do a bruteforce grid search and call it a day, you're most likely going to overfit your model to the data. This is what I've seen happen when amateurs (for lack of a better word) build ML systems:

(1) You'll get tremendously good accuracies on your training dataset with grid search (2) Business decisions will be made based on the high accuracy numbers you're seeing (90%? wow! we've got a helluva product here!) (3) The model will be deployed to production. (4) Accuracies will be much lower, perhaps 5-10% lower if you're lucky, perhaps a lot more. (5) Scramble to explain low accuracies, various heuristics put in place, ad-hoc data transforms, retrain models on new data -- all essentially groping in the dark, because now there's a fire and you can't afford the time to learn about model regularization and cross-validation techniques.

And eventually you'll have a patchwork of spaghetti that is perhaps ML, perhaps just heuristics mashed together. So while there's value in being practical, when ML becomes a commodity enough to be in an IT stack, it is likely no longer considered ML.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#53

Post by jostmey »

scottlocklin wrote: post #50
jostmey wrote: post #17

I am glad he pointed out that most artificial neural networks bear only a superficial resemblance to our own biological ones. But I think he failed to appreciate the power behind Boltzmann Machines - a type of neural network designed to create a generative model of a dataset. Personally, I find the resemblance between these neural networks and the real ones a little uncanny. And very few people seem to realize that t…

I've only started looking at RBMs recently, but ... what are you talking about? Biological neural networks use spikes. RBMs certainly don't. RBMs look more like HMMs to me than like biological neurons. Don't take this as me saying, "you're wrong" -I'm curious if there is another way to think about RBMs (aka, "papers please" -so I have a deeper understanding when I do my own implementation of RBMs).

The energy function for a Boltzmann machine is usually E = v^T x W x h, where v are the visible units, W contains the weights, and h represents the hidden units. The form of the energy function defines the activation function of each neuron and the learning rule that goes with it. Now, you can in theory start off with any energy function you like (this form just happens to be the simplest). You would then have to re-derive the activation function and learning rule.

Just what would the energy function look like for real neurons? I don't know but we do know that the activation function would have to "spike" in bursts. So that is a clue. We also have rudimentary ideas about the learning rule used in biological neural networks, so you would also want to take this into account when determining the actual energy function. Finally, real neurons do not send retrograde signals but are instead wired recurrently, which must also be taken into consideration.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#54

Post by arca_vorago »

I recently worked for a very cutting edge bioinformatics company, and I particularly agree with his segment about data sizes growing.

What I would say though, is that I think it is less an issue of the statistical strength of the data, and has more to do with the methods used to turn data itself into the statistics. For example, I was working with what by now (size projections are paramount in sysadmin planning for stuff like this) should be close to a Petabytes worth of genetic data. The real issue we were running into was that the traditional tools tend to fall apart on data of this size.

What we ended up doing was writing a distribution protocol for a certain application that worked well but wasn't very concurrent, and then every machine on the network besides the storage/sequencers/backup would crunch the data, helping even the big servers out. A big server would get 10-30 workers and a workstation would get 1-4. We turned 2 day analysis into 4 hour analysis.

And once we did the analysis, only one person, the company owner/genius, could decipher it.

I have to say, as a sysadmin, it was probably one of the most challenging and most educational positions I ever had. I actually enjoyed always being the only person in the room without a Phd.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#56

Post by robg »

blennon wrote: post #12
robg wrote: post #6

Earlier quoted context omitted.

The brain could, and likely does, individual neurons do not. Synaptic current only travels in one direction.

This is not entirely correct. There is a lot of evidence that action potentials backpropagate into the dendrites [1]. On a local level, this allows synapses on the dendrites to "be aware" of the activity of the post-synaptic neuron. However, I do not think this accomplishes/implements the ANN backpropagation algorithm (although a few statements and a citation in [1] alludes to this being possible). [1] http://en.wiki…

Yes, that is a very interesting complication to the standard model.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#57

Post by sushirain »

tormeh wrote: post #40

>In the brain, we have precious little idea how learning is actually taking place. It's Hebbian learning. When a post-synaptic neuron fires shortly after a pre-synaptic one fires, the synapse in question is strengthened (the surface area actually becomes larger). I hope he's talking about higher level concepts of learning, because otherwise he's wrong.

Hebbian learning is the little idea we do have. We don't know much more: how memories are represented by neurons, control, consciousness, vision, and almost anything.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#58

Post by 1wu »

elpachuco wrote: post #10

>>Another example of a good language problem is question answering, like “What’s the second-biggest city in California that is not near a river?” If I typed that sentence into Google currently, I’m not likely to get a useful response. So I typed that in google just to see and indeed I got nothing. I guess their [1]knowledge graph still has a long way to go. [1 http://www.google.com/insidesearch/features/search/knowle…

http://www.quora.com/What%E2%80%99s-the-second-biggest-city-...

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#59

Post by Houshalter »

sherjilozair wrote: post #22
conistonwater wrote: post #20

I learned about machine learning way after I learned mathematics, so it always amused me that back propagation = chain rule = forward differentiation = adjoint differentiation and that different disciplines have different words for what is just the chain rule.

None of the parties mentioned actually deny the above equivalence. The reason backprop is a popular idea in deep learning is because people started developing continuous models, where the output (and the error) was a continuous and differentiable function of the input and the weights, which allowed chain rule to be used to compute the gradients, which allowed one to use gradient descent methods. This shift from discr…

It's kind of unfortunate, as it's forced everything to be continuous. Which is not very computationally efficient or easily interpreted by humans.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#60

Post by tdullien »

michaelochurch wrote: post #42
muraiki wrote: post #38

Earlier quoted context omitted.

As someone who has tried out various MOOCs and entry level resources on machine learning, this is the same conclusion I came to. Beyond any sort of trivial example, I found I lacked the mathematical and statistical knowledge to not only interpret the results in a relatively unbiased and error-free way, but to know "what to do next." What scares me is that MOOCs are really pushing the data scientist field -- see Udaci…

Beyond any sort of trivial example, I found I lacked the mathematical and statistical knowledge to not only interpret the results in a relatively unbiased and error-free way, but to know "what to do next." The popular MOOCs don't take you far enough to start doing serious machine learning, but you don't need a PhD to be ready to solve those problems. It takes work. Lots of work. Re-learn linear algebra until you know…

What you are describing as a background is all part of a "normal" math/CS undergrad education (at least in Germany where I studied).

From a level of the mathematical difficulty, Elements etc. (but also current-level research papers) are all readable by anyone with a solid understanding of undergraduate mathematics (which is essentially a decent Linear Algebra course, multivariate analysis, a probability course, and a numerical computing course).

I think the reason why employers look for candidates with a PhD is that too many people "scrape by" when getting their CS degree -- e.g. they somehow fulfilled the required coursework, and somehow got their degree. The PhD requirement is essentially a bureaucratic substitute for answering the question "has this person understood math in sufficient depth to be able to do independent work with it".

Post reply on HN