Relevant but unmentioned in his list of accolades, Jordan is the 2015 Rumelhart prize winner, which is the equivalent of the Turing award for the Cognitive Sciences.
Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
11–20 of 60 posts
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#12These two comments seem to contradict each other: "we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like." "...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. " So w…
The brain could, and likely does, individual neurons do not. Synaptic current only travels in one direction.
This is not entirely correct. There is a lot of evidence that action potentials backpropagate into the dendrites [1]. On a local level, this allows synapses on the dendrites to "be aware" of the activity of the post-synaptic neuron. However, I do not think this accomplishes/implements the ANN backpropagation algorithm (although a few statements and a citation in [1] alludes to this being possible).
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#13>>Another example of a good language problem is question answering, like “What’s the second-biggest city in California that is not near a river?” If I typed that sentence into Google currently, I’m not likely to get a useful response. So I typed that in google just to see and indeed I got nothing. I guess their [1]knowledge graph still has a long way to go. [1 http://www.google.com/insidesearch/features/search/knowle…
Wolfram can handle it: http://www.wolframalpha.com/input/?i=2nd+biggest+city+in+Cal...
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#14>>Another example of a good language problem is question answering, like “What’s the second-biggest city in California that is not near a river?” If I typed that sentence into Google currently, I’m not likely to get a useful response. So I typed that in google just to see and indeed I got nothing. I guess their [1]knowledge graph still has a long way to go. [1 http://www.google.com/insidesearch/features/search/knowle…
Wolfram can handle it: http://www.wolframalpha.com/input/?i=2nd+biggest+city+in+Cal...
That misses the key qualifier: "near a river". The challenge there is what is "near", and what is a "river"?
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#15>>Another example of a good language problem is question answering, like “What’s the second-biggest city in California that is not near a river?” If I typed that sentence into Google currently, I’m not likely to get a useful response. So I typed that in google just to see and indeed I got nothing. I guess their [1]knowledge graph still has a long way to go. [1 http://www.google.com/insidesearch/features/search/knowle…
Wolfram can handle it: http://www.wolframalpha.com/input/?i=2nd+biggest+city+in+Cal...
Uh, no, that's just "the second biggest city" Wolfram can't even handle "What is the second biggest city in California near a river" - IE, can't do something one step beyond the trivial.
http://www.wolframalpha.com/input/?i=What+is+the+second+bigg...
Can't even handle "What is the second biggest city in northern California" http://www.wolframalpha.com/input/?i=What+is+the+second+bigg...
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#16A lot of people are building things [with big data] hoping that they work, and sometimes they will ... Eventually, we have to give real guarantees. Civil engineers eventually learned to build bridges that were guaranteed to stand up. So with big data, it will take decades, I suspect, to get a real engineering approach, so that you can say with some assurance that you are giving out reasonable answers and are quantifying the likelihood of errors.
It's seems like the idea is that machine learning and data driven inference have to grow up and become a real scientific discipline. "Why can't you be more like Civil Engineering?" This isn't the best way to look at it. Machine learning is designed for situations where data is limited and there are no guarantees. Take Amazon's recommendation engine for example. It's not possible to peer into someone's mind and come up with a mathematical proof that states whether they will like or dislike John Grisham novels. A data driven model can use inference to make predictions based on the person's rating history, demographic profile, etc. It's true that many machine learning approaches don't have the scientific heft of civil engineering, but they are still very useful in many situations.
I'm not disagreeing with the eminence of Michael I. Jordan. I think this is a philosophical question with no correct answer. Is the world deterministic, can we model everything with rigorous physics style equations? Or is it probabilistic, are we always making inferences based on a limited amount of data? Both of those views are valid, especially in different contexts. Some of the most interesting problems are inherently probabilistic, such as predicting the weather, economic trends and the behavior of our own bodies. "Big Data" is obviously a stupid buzzword, but the concept of data driven decision making is very sound. We should put less focus on media hype terms and continue to encourage people to make use of large amounts of information. Get rid of the bathwater, keep the baby.
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#17I am glad he pointed out that most artificial neural networks bear only a superficial resemblance to our own biological ones. But I think he failed to appreciate the power behind Boltzmann Machines - a type of neural network designed to create a generative model of a dataset. Personally, I find the resemblance between these neural networks and the real ones a little uncanny. And very few people seem to realize that the formalism behind a Boltzmann Machine can be adapted to fit the activation patterns of real neurons - you just have to redefine the energy function to match that of real biological neurons.
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#18A lot of people are building things [with big data] hoping that they work, and sometimes they will ... Eventually, we have to give real guarantees. Civil engineers eventually learned to build bridges that were guaranteed to stand up. So with big data, it will take decades, I suspect, to get a real engineering approach, so that you can say with some assurance that you are giving out reasonable answers and are quantify…
A lot of the talk on machine learning reminds me of thermodynamics. There are some states we can say can happen. We can determine what a state can be composed of in terms of microstates with certain probabilities. We have definite answers for some things, and in other situations, we have to settle for big picture images. It all depends on the measure of the space you are working in. Nevertheless, there are ways to quantify errors in machine learning routines. There are mathematically sound ways to reduce error too, and intuition gives us even more models (to test). I do not think it should be a debate based on deterministic and probabilistic guarantees. The question should be more geared to how can we make assumptions to form better models and consistently do testing along the way.
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#19A lot of people are building things [with big data] hoping that they work, and sometimes they will ... Eventually, we have to give real guarantees. Civil engineers eventually learned to build bridges that were guaranteed to stand up. So with big data, it will take decades, I suspect, to get a real engineering approach, so that you can say with some assurance that you are giving out reasonable answers and are quantify…
You misunderstood what he is saying.
> Similarly here, if people use data and inferences they can make with the data without any concern about error bars, about heterogeneity, about noisy data, about the sampling pattern, about all the kinds of things that you have to be serious about if you’re an engineer and a statistician—then you will make lots of predictions, and there’s a good chance that you will occasionally solve some real interesting problems. But you will occasionally have some disastrously bad decisions. And you won’t know the difference a priori. You will just produce these outputs and hope for the best.
He is not saying anything about the relative heft of machine learning and civil engineering. He is saying that if you don't worry about whether your predictions coming from big data are accurate, and whether you know a priori that they are accurate, you will still make predictions, but some of them will be wrong, and you don't know which ones. The analogy with engineering is only incidental to his point, which is mainly about overfitting.
You can point out afterwards that a certain prediction made using big data was correct in hindsight by collecting data after the prediction was used to make some decisions, like Amazon might. But you would really like to know whether a decision is likely to be a good one before you make it. And he, as a scientist, is interested in knowing for sure whether his results are correct.
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#20I learned about machine learning way after I learned mathematics, so it always amused me that
back propagation = chain rule = forward differentiation = adjoint differentiation
and that different disciplines have different words for what is just the chain rule.