Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

spectrum.ieee.org

Post reply on HN
Print view View original

31–40 of 60 posts

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#31

Post by jimduk »

sherjilozair wrote: post #23
mathgenius wrote: post #3

These two comments seem to contradict each other: "we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like." "...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. " So w…

This is a reply to multiple sibling comments. There is actually recent work which shows that deep learning methods can also work WITHOUT any reverse signals: http://s.yosinski.com/dan_cownden_presentation.pdf

Interesting paper. Any more details on the architecture of the feedback connections. Also I can't tell from the paper where and how weights are being updated e..g what does "train" mean in this context

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#32

Post by fhnjh »

Daishiman wrote: post #24

Great interview. In my experience it's amazing just how many people are talking about "Big Data" and just how exactly none of those are the ones with the necessary PhDs in statistics and algorithms to get anything of any value done. In my experience there are very few domains within Machine Learning where you don't need to be an expert in the field to yield useful conclusions out of the data. Even if you have a high-…

I agree, machine learning currently requires huge globs of knowledge to wield. As such it is a tool for an intelligent entity to use, it is not intelligent in it's own right. I disagree with Michael about the correct way forward for machine learning, the problem is not one of piecemeal engineering. If you are spending your time developing algorithms to solve a particular class of problem, you are wasting your time(in terms of pursuit of GAI). The overarching problem needs to tackled, what is the correct framework within which to think about intelligent systems? This is, as mentioned in the article a question of insight - we need the metaphorical apple to fall on some bright sparks head. But if all the bright sparks are fully engaged in chasing the short term problem it might take a long time.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#34

Post by cdoxsey »

diydsp wrote: post #29

I was grateful and surprised to see the article start off immediately with a meta-remark on the collusion between pop science media and academics. It recalled one my frustrations during grad school in the late 2000s: student researchers striving for recognition and journalists sexing up our stories that misinformed the public. This feedback loop explains a great chunk of why we on HN spend so much time knit-picking t…

All news is like that. When they cover stories we actually know something about we see that its all misinformed BS, but then for some strange reason, on other issues, we're perfectly happy to have every thing voxplained to us. (or the NY times is gospel if that floats your boat) Michael Chrichton:

“Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.”

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#35

Post by tempodox »

Indeed, the big-data winter is just waiting to happen, after all the hot air that has been produced (& continues to be). Anyway, it's very nice to see the media hype put in perspective for a change.

I can see how some people might feel like being between a rock and a hard place: The data firehoses are all in place, our key-value stores are getting fuller by the hour, and we're supposed to sit and wait for decades before we'll be able to make any sense of it? I wouldn't be surprised if some will much rather play roulette today than make a sure bet in 10+ yrs.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#36

Post by fear91 »

fhnjh wrote: post #32
Daishiman wrote: post #24

Great interview. In my experience it's amazing just how many people are talking about "Big Data" and just how exactly none of those are the ones with the necessary PhDs in statistics and algorithms to get anything of any value done. In my experience there are very few domains within Machine Learning where you don't need to be an expert in the field to yield useful conclusions out of the data. Even if you have a high-…

I agree, machine learning currently requires huge globs of knowledge to wield. As such it is a tool for an intelligent entity to use, it is not intelligent in it's own right. I disagree with Michael about the correct way forward for machine learning, the problem is not one of piecemeal engineering. If you are spending your time developing algorithms to solve a particular class of problem, you are wasting your time(in…

What makes you think human brain isn't just an ensemble of hundreds of different specialized algorithms?

Trying to emulate biological brains might not be the way forward. People tried to fly by constructing bird-like feathers and wings - it obviously didn't work. We had to understand the underlying principles governing flight. The same applies to creating neural networks.

There's some underlying principle the brain uses. It doesn't mean we have to crack brain structure to achieve strong intelligence.

We should look for inspiration in biological systems but we should not try to copy them.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#37

Post by murbard2 »

fear91 wrote: post #36
fhnjh wrote: post #32

Earlier quoted context omitted.

I agree, machine learning currently requires huge globs of knowledge to wield. As such it is a tool for an intelligent entity to use, it is not intelligent in it's own right. I disagree with Michael about the correct way forward for machine learning, the problem is not one of piecemeal engineering. If you are spending your time developing algorithms to solve a particular class of problem, you are wasting your time(in…

What makes you think human brain isn't just an ensemble of hundreds of different specialized algorithms? Trying to emulate biological brains might not be the way forward. People tried to fly by constructing bird-like feathers and wings - it obviously didn't work. We had to understand the underlying principles governing flight. The same applies to creating neural networks. There's some underlying principle the brain u…

> What makes you think human brain isn't just an ensemble of hundreds of different specialized algorithms?

Experiments such as this one http://web.mit.edu/msur/www/publications/Newton_Sur04.pdf

The plasticity of the brain is phenomenal.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#38

Post by muraiki »

Daishiman wrote: post #24

Great interview. In my experience it's amazing just how many people are talking about "Big Data" and just how exactly none of those are the ones with the necessary PhDs in statistics and algorithms to get anything of any value done. In my experience there are very few domains within Machine Learning where you don't need to be an expert in the field to yield useful conclusions out of the data. Even if you have a high-…

As someone who has tried out various MOOCs and entry level resources on machine learning, this is the same conclusion I came to. Beyond any sort of trivial example, I found I lacked the mathematical and statistical knowledge to not only interpret the results in a relatively unbiased and error-free way, but to know "what to do next."

What scares me is that MOOCs are really pushing the data scientist field -- see Udacity and Coursera -- but giving people only enough knowledge to be dangerous. It's tough because data science is a fascinating field and many people who have the interest and aptitude don't have the means or life situation to go to grad school for it. These MOOCs are trying to appeal to such people, but they're not nearly rigorous enough.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#39

Post by blauwbilgorgel »

Daishiman wrote: post #24

Great interview. In my experience it's amazing just how many people are talking about "Big Data" and just how exactly none of those are the ones with the necessary PhDs in statistics and algorithms to get anything of any value done. In my experience there are very few domains within Machine Learning where you don't need to be an expert in the field to yield useful conclusions out of the data. Even if you have a high-…

> ... how exactly none of those are the ones with the necessary PhDs in statistics and algorithms to get anything of any value done.

I see it almost the other way around: Companies strictly demand PhD's for Big Data jobs and can't find this unicorn. Yet we live in a time where we don't need a PhD program to receive education from the likes of Ng, LeCun and Langford. We live in a time where curiosity and dedication can net you valuable results. Where CUDA-hackers can beat university teams. The entire field of big data visualization requires innate aptitude and creativity, not so much an expensive PhD program. I suspect Paul Graham, when solving his spam problem with ML, benefited more from his philosophy education than his computer science education.

Of course, having a PhD. still shows dedication and talent. But it is no guarantee for practical ML skills, it can even hamper research and results, when too much power is given to theory and reputation is at stake.

In my experience Machine Learning was locked up in academics, and even in academics it was subdivided. The idea that "you need to be an ML expert, before you can run an algo" is detrimental to the field, not helping so much in adopting a wider industry use of ML. Those ML experts set the academic benchmarks that amateurs were able to beat by trying out Random Forests and Gradient Boosting.

I predict that ML will become part of the IT-stack, as much as databases have. Nowadays, you do not need to be a certified DBA to set up a database. It is helpful and in some cases heavily advisable, but databases now see a much wider adoption by laypeople. This is starting to happen in ML. I think more hobbyists are right now toying with convolutional neural networks, than there are serious researchers in this area. These hobbyists can surely find and contribute valuable practical insights.

Tuning parameters is basically a gridsearch. You can bruteforce this. In goes some ranges of parameters, out come the best params found. Fairly easy to explain to a programmer.

Adapting existing algorithms is ML researcher territory. That is a few miles above the business people extracting valuable/actionable insight from (big or small or tedious) data. Also there is a wide range of big data engineers making it physically possible to have the "necessary" PhD's extract value from Big Data.

Re: Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others

#40

Post by tormeh »

>In the brain, we have precious little idea how learning is actually taking place.

It's Hebbian learning. When a post-synaptic neuron fires shortly after a pre-synaptic one fires, the synapse in question is strengthened (the surface area actually becomes larger). I hope he's talking about higher level concepts of learning, because otherwise he's wrong.

Post reply on HN