Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
1–10 of 60 posts
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#2> When you have large amounts of data, your appetite for hypotheses tends to get even larger. And if it’s growing faster than the statistical strength of the data, then many of your inferences are likely to be false. They are likely to be white noise.
It's actually worse than that. What I see is that when companies have the ability to store and "analyze" large amounts of data, their appetite for data tends to increase. So they seek to take in as much data as they can find. More often than not, the quality of the data is mixed at best. Frequently, it's horrible, and because the focus is on data acquisition and not data quality, nobody notices the bad data, missing data and duplicate data.
The result: even if you manage to come up with decent hypotheses, you can't trust the data on which you test them.
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#3These two comments seem to contradict each other:
"we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like."
"...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. "
So why can't the brain do some kind of backpropagation?
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#4These two comments seem to contradict each other: "we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like." "...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. " So w…
I agree. Neuroscience isn't my specialty, but I don't remember hearing about research showing that the brain does not use feedback...
Overall though, I think there is a lot of truth to his message.
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#5These two comments seem to contradict each other: "we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like." "...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. " So w…
We know the brain doesn't do "backpropogation" in the sense that backpropagation is a complete description of how the brain works. It may be a component, but there has to be much more, because what backpropogation can do is fairly well characterized mathematically now and is not sufficient to build anything like a human brain. If you haven't seen a paper that clearly spells that out, it's because it would be considered too trivially obvious a result to publish.
When we don't know how a thing works that does not preclude us being able to eliminate some of the possibilities. Our ignorance is profound, but not total.
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#6These two comments seem to contradict each other: "we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like." "...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. " So w…
The brain could, and likely does, individual neurons do not. Synaptic current only travels in one direction.
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#7This was an incredibly rational interview.
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#8These two comments seem to contradict each other: "we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like." "...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. " So w…
I agree. Neuroscience isn't my specialty, but I don't remember hearing about research showing that the brain does not use feedback... Overall though, I think there is a lot of truth to his message.
Think about it; biological neurons most definitely don't use backprop. Physical neurons can't backprop linearly if they forward prop nonlinearly. I think Hinton pointed this out, though almost all ANN researchers agree.
Hebbian learning .... maybe. I never thought too much about contrastive divergence.
It doesn't really matter; brains most certainly don't work like ANNs, except maybe in some weird mean field sense for a few things like liquid state machines. It would be a huge coincidence if LeCun or Hinton or whoever magically wrote down the brain equation....
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#9These two comments seem to contradict each other: "we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like." "...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. " So w…
We know the brain doesn't do "backpropogation" in the sense that backpropagation is a complete description of how the brain works. It may be a component, but there has to be much more, because what backpropogation can do is fairly well characterized mathematically now and is not sufficient to build anything like a human brain. If you haven't seen a paper that clearly spells that out, it's because it would be consider…
[deleted]
Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others
#10>>Another example of a good language problem is question answering, like “What’s the second-biggest city in California that is not near a river?” If I typed that sentence into Google currently, I’m not likely to get a useful response.
So I typed that in google just to see and indeed I got nothing. I guess their [1]knowledge graph still has a long way to go.
[1http://www.google.com/insidesearch/features/search/knowledge...]