Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning has become alchemy (2017) [video]

youtube.com

Post reply on HN
Print view View original

101–110 of 134 posts

Re: Machine learning has become alchemy (2017) [video]

#101

Post by bartread »

joe_the_user wrote: post #3

It is simple to say deep learning is based on "alchemy" or "engineering" or whatever it is that isn't strong theory. And it's reasonable to say deep learning has a lot of mathematical and statistical intuitions but doesn't have a strong theory - maybe just doesn't yet have a strong theory or maybe can never get one. So this is by now a standard argument. The standard answers I think have been: 1) Well, we are discove…

Worth also bearing in mind that we've been here before in other fields. Alchemy ultimately became chemistry.

Even in the Victorian era when, for fairly large swathes of the periodic table, and different types of compound, we already had a quite good experimental understanding of chemical reactions in terms of their constituent components and products, along with the conditions under which those reactions occur, we still didn't know the why. We didn't understand much about atoms or how they bond together, for example.

The point is this: science can often take a long time to advance, and AI is still a very young field, with the first practical endeavours only dating back to the post-WWII period.

Should we therefore be terribly surprised that ML seems a bit like alchemy?

As an aside, another normal facet of scientific advancement is the vast quantity of naysayers encountered along the way. Haters gonna hate, I suppose. (But don't misunderstand me: whilst I'm not an ML fanboi, I recognise that advances come in fits and starts, dead-ends will be encountered, and overall it's going to take quite a long time and require a lot of hard work to get anywhere.)

Final aside: this video has definitely been posted here before but I've also been unable to find it.

Re: Machine learning has become alchemy (2017) [video]

#102

Post by baxtr »

Barrin92 wrote: post #21

The problem with ML in my opinion is not that we're missing some sort of fundamental theory, but that there simply is none. ML is essentially fancy pattern matching roughly resembling the human visual system, which is why it happens to be good at tasks related to perception. It's not some master algorithm, it's not going to produce sci-fi AI, and it probably isn't even suited to solve most problems in the realm of in…

The moment I realized that Deep Learning is nothing more than a non-linear matrix operation I lost a big deal of respect for the field. I still believe the potential of Deep Learning is huge, since many, especially larger companies, have lots of data that just sits there waiting for some innovation. DL can deliver more productivity, i.e. higher quality, faster processes etc. That is great. But this has very little to do with the fancy sci-fi version of AI.

Re: Machine learning has become alchemy (2017) [video]

#103

Post by Farradfahren »

Barrin92 wrote: post #21

The problem with ML in my opinion is not that we're missing some sort of fundamental theory, but that there simply is none. ML is essentially fancy pattern matching roughly resembling the human visual system, which is why it happens to be good at tasks related to perception. It's not some master algorithm, it's not going to produce sci-fi AI, and it probably isn't even suited to solve most problems in the realm of in…

Actually it could produce SciFi AI - if combined with evolutionary algos. Just look at how every toddler does it- act and produce a lot of data, then check for signal -> reaction reproduceability. Once that is stable, fine tune. Then use the new gained abilitys to repeat the process.

Re: Machine learning has become alchemy (2017) [video]

#104

Post by [deleted] »

TrackerFF wrote: post #55

The ML scene was more rigorous 10-15-20 years ago, because it was mostly confined to the world of academia and industrial R&D, and we had much less wide-reaching problems to work towards. As tech evolves, more data is being generated, which in turn creates more problems that increases the demand for solutions. To put it short: Solving "real world" problems is more/better rewarded than figuring out the underlying tech…

[deleted]

Re: Machine learning has become alchemy (2017) [video]

#105

Post by andreyk »

baxtr wrote: post #102
Barrin92 wrote: post #21

The problem with ML in my opinion is not that we're missing some sort of fundamental theory, but that there simply is none. ML is essentially fancy pattern matching roughly resembling the human visual system, which is why it happens to be good at tasks related to perception. It's not some master algorithm, it's not going to produce sci-fi AI, and it probably isn't even suited to solve most problems in the realm of in…

The moment I realized that Deep Learning is nothing more than a non-linear matrix operation I lost a big deal of respect for the field. I still believe the potential of Deep Learning is huge, since many, especially larger companies, have lots of data that just sits there waiting for some innovation. DL can deliver more productivity, i.e. higher quality, faster processes etc. That is great. But this has very little to…

While it's true that the basics of Deep Learning (eg ImageNet) are not that conceptually interesting, it is also worth noting people are working on tons of interesting directions within this framework combined with other ideas from AI (see eg 'Learning to Reason with Third-Order Tensor Products' - https://arxiv.org/abs/1811.12143 | 'Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs' - https://arxiv.org/abs/1812.02788)

Re: Machine learning has become alchemy (2017) [video]

#106

Post by RA_Fisher »

typon wrote: post #25
Barrin92 wrote: post #21

The problem with ML in my opinion is not that we're missing some sort of fundamental theory, but that there simply is none. ML is essentially fancy pattern matching roughly resembling the human visual system, which is why it happens to be good at tasks related to perception. It's not some master algorithm, it's not going to produce sci-fi AI, and it probably isn't even suited to solve most problems in the realm of in…

The annoying part is that ML is not sold to the world like this. I would say the truth (I believe) in this comment is the "dirty little secret" of our industry. Everyone working on it knows this but the research and VC dollars are flowing in so no one wants to talk about it too much.

Statistics is quite valuable and grounded in theory. It's a trusty tool if used with respect (like a knife).

Re: Machine learning has become alchemy (2017) [video]

#107

Post by RA_Fisher »

denzil_correa wrote: post #99
another-one-off wrote: post #85

Earlier quoted context omitted.

> Would you consider them trusthworthy in court, where lives are at stake? Probably. Human intelligence is extremely fallible - based on the statistics the only reason we trust humans to do half the stuff they do is because there is literally no choice. If we held humans to a high objective engineering standard We wouldn't: * Let them drive * Let them present their memories as evidence in a court case * Entrust them…

> Basically, you might be dead on the money that they are not 'trustworthy enough', but lets not lose sight of the fact that even being an order of magnitude from human performance might be enough after costs and engineering benefits get factored in. Ironically, the thing that is lost in this comment would be "accountability". In case of a human, you can go back / trace decision making criteria and hold someone accou…

Linear models are highly interpretable and an operator can be held accountable.

Re: Machine learning has become alchemy (2017) [video]

#108

Post by RA_Fisher »

jsinai wrote: post #86
laretluval wrote: post #30

Earlier quoted context omitted.

There’s going to be people plugging their ears and shouting “but it’s just nonlinear function approximation!” all the way into the singularity.

Classic regression is only intelligible because there were only a few parameters and people could use ANOVA to try and interpret them. IMO ANOVA is alchemy as well and most people trained to use it, don’t fully understand it. Should we leave decisions to those kinds of models? Not to mention that part of what makes NN such a step forward is precisely the high nonlinearity. When you have millions of parameters, the co…

ANOVA is grounded by theory that has stood the test of time for decades. Statisticians that question ANOVA aren't a thing (despite such an accurate criticism being valuable career-wise to an academic). Gelman points out that multilevel models are easier to use than ANOVA and I tend to agree.

Re: Machine learning has become alchemy (2017) [video]

#109

Post by RA_Fisher »

notafraudster wrote: post #67
tnecniv wrote: post #59

Earlier quoted context omitted.

I believe OP is speculating about state-of-the-art ML. The classical techniques, like linear regression, are well studied. However, at this point, there is a race to explain theoretically _why_ deep learning is so successful at generalization when, by classical standards, it shouldn't be. OP is speculating that this quest is in vain.

I agree with your interpretation and pushback against your parent comment, but wanted to elaborate that the "problem" here is much deeper. Deep learning is basically magic, okay, fine, so let's start much closer to linear regression. Say we have a predictor matrix X with 2 predictors. We fit a model using a penalized linear regression (say LASSO) adding to our predictor matrix an interaction terms, arbitrary polynomi…

You're spot on. Another point I like to raise is that predictors are often extremely redundant in terms of information. The implication being that nearly as good predictions might be made with three predictors vs thousands.

Re: Machine learning has become alchemy (2017) [video]

#110

Post by RA_Fisher »

nurettin wrote: post #57
QML wrote: post #41

Earlier quoted context omitted.

Wait, what? Are you confusing ML with AI? The last two courses I took on ML were all theory. For example, when we use linear regression, we make assumptions about the noise — i.i.d. and Gaussian.

Would you describe AI as: automatic categorization and decision making? If so, then Machine Learning is a part of modeling AI. Regardless of how they are taught in terms of University lectures.

I'd describe AI/ML as statistics by another name.
Post reply on HN