Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning has become alchemy (2017) [video]

youtube.com

Post reply on HN
Print view View original

91–100 of 134 posts

Re: Machine learning has become alchemy (2017) [video]

#91

Post by kuu »

Barrin92 wrote: post #21

The problem with ML in my opinion is not that we're missing some sort of fundamental theory, but that there simply is none. ML is essentially fancy pattern matching roughly resembling the human visual system, which is why it happens to be good at tasks related to perception. It's not some master algorithm, it's not going to produce sci-fi AI, and it probably isn't even suited to solve most problems in the realm of in…

ML != Deep Learning.

Deep Learning is just a branch (or even a sub-branch inside NN) of many ML techniques. Not all of them suffer the same problems or are based in the same theoretical background.

Re: Machine learning has become alchemy (2017) [video]

#92

Post by [deleted] »

tnecniv wrote: post #59
QML wrote: post #41

Earlier quoted context omitted.

Wait, what? Are you confusing ML with AI? The last two courses I took on ML were all theory. For example, when we use linear regression, we make assumptions about the noise — i.i.d. and Gaussian.

I believe OP is speculating about state-of-the-art ML. The classical techniques, like linear regression, are well studied. However, at this point, there is a race to explain theoretically _why_ deep learning is so successful at generalization when, by classical standards, it shouldn't be. OP is speculating that this quest is in vain.

[deleted]

Re: Machine learning has become alchemy (2017) [video]

#93

Post by EvilTerran »

notafraudster wrote: post #61
QML wrote: post #41

Earlier quoted context omitted.

Wait, what? Are you confusing ML with AI? The last two courses I took on ML were all theory. For example, when we use linear regression, we make assumptions about the noise — i.i.d. and Gaussian.

In the context of linear regression, there is no particular reason to assume noise is i.i.d. and Gaussian. The former is part of the the Gauss-Markov conditions, under which OLS is BLUE (the "best linear unbiased estimator"). The latter is not necessary at all. And of the Gauss-Markov conditions can be violated to varying degrees of consequence. In fact, in real data, these assumptions are almost always violated. The…

For the benefit of laypeople like myself:

- IID: "independent and identically distributed", https://en.wikipedia.org/wiki/Independent_and_identically_di...

- OLS: "ordinary least squares", https://en.wikipedia.org/wiki/Ordinary_least_squares (I think)

Re: Machine learning has become alchemy (2017) [video]

#94

Post by make3 »

Barrin92 wrote: post #21

The problem with ML in my opinion is not that we're missing some sort of fundamental theory, but that there simply is none. ML is essentially fancy pattern matching roughly resembling the human visual system, which is why it happens to be good at tasks related to perception. It's not some master algorithm, it's not going to produce sci-fi AI, and it probably isn't even suited to solve most problems in the realm of in…

there is huge with being done right now on the theoretical backing of deep learning.

classical machine learning has huge, sound theoretical backing.

scaling well is extremely important, and is the main advantage of deep learning. as planes don't need a bird's feathers to be useful, deep learning is already way sufficiently successful enough to justify their use.

no one serious claims that deep nets are biologically possible. they are still promising, and useful, powering translation, phone keyboard autocomplete, search, camera stuff like low light mode and cell phone portait mode, medical imaging diagnosis, malware detection, new planet detection in astronomy, plasma modeling in energy engineering, etc

Re: Machine learning has become alchemy (2017) [video]

#95

Post by perfmode »

another-one-off wrote: post #85
turingspiritfly wrote: post #31

Earlier quoted context omitted.

Those massive gains have yet to considered reliable enough to be considered trusthworthy. Would you consider them trusthworthy in court, where lives are at stake? Gains are nice but we are still so far from the essence of AI systems and considering how much resources we are pouring into learning, at this point all of them appear as nothing more than massive fat expensive toys

> Would you consider them trusthworthy in court, where lives are at stake? Probably. Human intelligence is extremely fallible - based on the statistics the only reason we trust humans to do half the stuff they do is because there is literally no choice. If we held humans to a high objective engineering standard We wouldn't: * Let them drive * Let them present their memories as evidence in a court case * Entrust them…

https://www.usenix.org/conference/usenixsecurity18/presentat...

Re: Machine learning has become alchemy (2017) [video]

#96

Post by groestl »

berkut wrote: post #69
Barrin92 wrote: post #21

The problem with ML in my opinion is not that we're missing some sort of fundamental theory, but that there simply is none. ML is essentially fancy pattern matching roughly resembling the human visual system, which is why it happens to be good at tasks related to perception. It's not some master algorithm, it's not going to produce sci-fi AI, and it probably isn't even suited to solve most problems in the realm of in…

I'd agree with this assessment 100% - ML can achieve some very impressive results - in my domain (computer graphics for movie VFX) with regards to image generation / identification / modification, e.g. denoising for the latter - however the result is almost always effectively a black box of functionality which: A: isn't always understood what it is actually doing or how. B: often needs to be retrained when input data…

I like that all of the issues you bring up around ML can be read as applying to a human expert/craftsperson as well! (still with a wider margin, of course)

Re: Machine learning has become alchemy (2017) [video]

#97

Post by EmbarrassedFuel »

quanticle wrote: post #90
soared wrote: post #27

Earlier quoted context omitted.

> It barely learns anything given the amount of computation effort an data that goes into it, but it just happens to be good enough to be practically preferable to old symbolic systems. I don’t follow this. Are you implying there haven’t been absolutely massive gains in computer vision, nlg, nlp, etc?

Are you implying there haven’t been absolutely massive gains in computer vision, nlg, nlp, etc? The implication is that the massive gains haven't been the result of any algorithmic breakthroughs, but rather have been due to the application of massive amounts of computational resources, which weren't available when current ML algorithms were invented. So far as I can tell, that's a true accusation. If you look at the…

A small bone of contention: it’s not about Moore’s law per se, which has been ‘dead’ since about 2013, coincidentally when the deep learning revival started. It’s matrix multiplication ASIC development that is driving the progress.

GPUs already existed when the idea to use them to make the feasible size of neural nets larger came about. For a long time the drive for the increase of GPU compute power was still gaming/commercial graphics houses. It’s really only in the last 1-2 years that we’ve seen highly specialised GPUs with features like tensor cores (or indeed google’s TPUs).

Also, calling neural nets ‘brute force’ because they use a lot of computing power to train a model is slightly reductionist - a true brute force approach to image recognition, ie enumerating all possible combinations of, say, 200x200x256x3 pixels, would be completely absurd and probably exceed the computing power currently available on earth.

Re: Machine learning has become alchemy (2017) [video]

#98

Post by mtrimpe »

wetpaws wrote: post #58
perl4ever wrote: post #36

Earlier quoted context omitted.

Human intelligence involves choosing which context to use at any given point. So far, so called AI appears to take an external source of context for granted, which is why it seems to me fundamentally different from "real" intelligence, just like your eyes are different from your entire nervous system.

Every time there is some progress in AI people would yell "but this is not a real AI 'cause humans can..." ( play go/understand pictures/language/semantics/etc, goalpost is always in motion)

AI carves away what it means to be human one trivial slice at a time.

That's how it always has been and always will be until perhaps one day there is truly nothing 'truly human' left to slice off.

Re: Machine learning has become alchemy (2017) [video]

#99

Post by denzil_correa »

another-one-off wrote: post #85
turingspiritfly wrote: post #31

Earlier quoted context omitted.

Those massive gains have yet to considered reliable enough to be considered trusthworthy. Would you consider them trusthworthy in court, where lives are at stake? Gains are nice but we are still so far from the essence of AI systems and considering how much resources we are pouring into learning, at this point all of them appear as nothing more than massive fat expensive toys

> Would you consider them trusthworthy in court, where lives are at stake? Probably. Human intelligence is extremely fallible - based on the statistics the only reason we trust humans to do half the stuff they do is because there is literally no choice. If we held humans to a high objective engineering standard We wouldn't: * Let them drive * Let them present their memories as evidence in a court case * Entrust them…

> Basically, you might be dead on the money that they are not 'trustworthy enough', but lets not lose sight of the fact that even being an order of magnitude from human performance might be enough after costs and engineering benefits get factored in.

Ironically, the thing that is lost in this comment would be "accountability". In case of a human, you can go back / trace decision making criteria and hold someone accountable. In case of an algorithm, everyone washes their hands off. Performance is not the only criteria to make a decision if algorithms are "trustworthy" over humans.

Re: Machine learning has become alchemy (2017) [video]

#100

Post by logicallee »

Barrin92 wrote: post #21

The problem with ML in my opinion is not that we're missing some sort of fundamental theory, but that there simply is none. ML is essentially fancy pattern matching roughly resembling the human visual system, which is why it happens to be good at tasks related to perception. It's not some master algorithm, it's not going to produce sci-fi AI, and it probably isn't even suited to solve most problems in the realm of in…

> It barely learns anything given the amount of computation effort and data that goes into it

You know what else barely learns anything given the amount of computation that goes into it? A kid.

With 100 billion neurons -- so on the order of a hundred gigabytes of RAM -- after 72 months of closely supervised learning they're still far from being able to do many rudimentary tasks. It takes over 10 years of training to do that.

Something tells me most ML researchers wouldn't be too unhappy with such an awful performance.

Post reply on HN