Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Limits of Machine Learning

nautil.us

Post reply on HN
Print view View original

11–20 of 48 posts

Re: The Limits of Machine Learning

#11

Post by j1vms »

wrong_variable wrote: post #4

How is this article on nautil.us ? Did the author just read the wikipedia.com page on Machine Learning ? There is an entire field in ML called unsupervised learning. Labelling data that do not have labels attached to them. Its not a Fundamental (uggh) limit, I am not sure if the author even knows what Fundamental means, its not like the halting problem, or heat death of the universe. ML is also a very young field wit…

In popular discussion, Deep Learning has unfortunately become synonymous with Machine Learning, which also, as of late, has itself become synonymous with AI.

Now, this might sound facetious, but to help avert another AI winter (maybe 10 years down the road), we need to be loud and vocal, educating at the very least the investing strata of society as to hyponymy/hypernymy relationships between these terms.

Re: The Limits of Machine Learning

#12

Post by graycat »

> The Fundamental Limits of Machine Learning

Depends on the largely mathematical assumptions can bring to the data. What can be done with the variety of assumptions is illustrated with details beyond belief in the QA section of most research libraries.

For more, the OP has

> Almost all of the learning we expect our computers to do—and much of the learning we ourselves do —is about reducing information to underlying patterns, which can then be used to infer the unknown.

Ah, NOW I see! The OP has stated a relatively narrow problem.

E.g., consider arrivals at HN: Over each 30 minutes or so, they about have to be a sample path of a Poisson process. Why? The renewal theorem, as in W. Feller's second volume. Can say that without looking at "patterns" in the data, indeed, without looking at any data at all.

Then from knowing that the arrivals are a Poisson process, there is a nice stream of results can get right away, without the data and even more with the data. E.g., the sum of two independent Poisson processes is another Poisson process. Then more generally can have a continuous time, discrete state space Markov process subordinated to that or a related Poisson process. From that can have some, say, network queuing calculations good for capacity planning, optimization of capacity planning, stochastic optimal control, anomaly detection, etc. Have a good shot at using the strong law of large numbers and the martingale convergence theorem.

Can say nearly all of this, and more, without looking for "patterns" in the data or looking at the data at all. Again, looking at the data can say still more.

There's a lot in the QA section of the library!

Re: The Limits of Machine Learning

#13

Post by tree_of_item »

wrong_variable wrote: post #4

How is this article on nautil.us ? Did the author just read the wikipedia.com page on Machine Learning ? There is an entire field in ML called unsupervised learning. Labelling data that do not have labels attached to them. Its not a Fundamental (uggh) limit, I am not sure if the author even knows what Fundamental means, its not like the halting problem, or heat death of the universe. ML is also a very young field wit…

Christopher Columbus didn't discover "an entire new world", he was just one of the first Europeans to land on a continent that had already been there, with plenty of people, for thousands of years.

Re: The Limits of Machine Learning

#14

Post by [deleted] »

tree_of_item wrote: post #13
wrong_variable wrote: post #4

How is this article on nautil.us ? Did the author just read the wikipedia.com page on Machine Learning ? There is an entire field in ML called unsupervised learning. Labelling data that do not have labels attached to them. Its not a Fundamental (uggh) limit, I am not sure if the author even knows what Fundamental means, its not like the halting problem, or heat death of the universe. ML is also a very young field wit…

Christopher Columbus didn't discover "an entire new world", he was just one of the first Europeans to land on a continent that had already been there, with plenty of people, for thousands of years.

[deleted]

Re: The Limits of Machine Learning

#15

Post by Animats »

Right, not too helpful.

Also, the machine shown in the picture isn't even a computer. It was a special-purpose machine used to read microfilms of mark-sense Census forms and write the results on tape. (I once had a summer job at Census HQ in Suitland MD, and saw the FOSDIC machine.)

There are fundamental limits to hill-climbing. So far, nobody has something that just keeps running and continues to get better. Hill-climbing maxes out after a while and stalls.

We still need another big idea after deep learning and machine learning in its present form. No idea where that will come from. Anybody see anything on the horizon?

Re: The Limits of Machine Learning

#16

Post by Animats »

mturmon wrote: post #8

Note to casual commenters: the precise real-world implications of the NFL theorems proved by Wolpert and collaborators have been difficult to appreciate, even to people well-versed in the computational learning world. Starting point: http://www.santafe.edu/media/workingpapers/12-10-017.pdf where we read: "However, arguably, much of that research has missed the most important implications of the theorems."

Is that a corollary of the information theory theorem that for any lossless compressed representation, there must some data pattern for which the compressed representation is bigger?

Re: The Limits of Machine Learning

#17

Post by pron »

wrong_variable wrote: post #4

How is this article on nautil.us ? Did the author just read the wikipedia.com page on Machine Learning ? There is an entire field in ML called unsupervised learning. Labelling data that do not have labels attached to them. Its not a Fundamental (uggh) limit, I am not sure if the author even knows what Fundamental means, its not like the halting problem, or heat death of the universe. ML is also a very young field wit…

1. Just because someone decided to use the words "intelligence" and "neural" when describing a class of statistical clustering algorithms often based on backpropagation of errors doesn't mean these algorithms have anything to do with the brain or intelligence, and if they do, the relationship is not necessarily direct and immediate. Speaking about the two as if the connection is clear only muddles our understanding. It's important to remember that these terms are meant to capture the imagination, not to describe scientific knowledge.

2. Even if there are proven limitations to those algorithms, and even if those algorithms are related to human intelligence, so what? We're not sure what intelligence is, and it is easy to show that it is not "a general ability to solve problems". Humans are great at solving some problems and pretty terrible at others. Obviously, the "intelligence algorithm" (whatever that means) has some serious limitations, and is not so good at some things. For example, human intelligence doesn't seem helpful in approximating solutions to computationally hard problems. It is obvious that intelligence (or any algorithm) has its limitations.

3. Understanding the limits of a field and having optimism are two separate things. A few good impossibility (or infeasibility) theorems help serve as a map, so you can be optimistic while knowing a bit more about your surroundings, rather than being optimistic while fumbling in the dark.

Re: The Limits of Machine Learning

#19

Post by ankurdhama »

j1vms wrote: post #11
wrong_variable wrote: post #4

How is this article on nautil.us ? Did the author just read the wikipedia.com page on Machine Learning ? There is an entire field in ML called unsupervised learning. Labelling data that do not have labels attached to them. Its not a Fundamental (uggh) limit, I am not sure if the author even knows what Fundamental means, its not like the halting problem, or heat death of the universe. ML is also a very young field wit…

In popular discussion, Deep Learning has unfortunately become synonymous with Machine Learning, which also, as of late, has itself become synonymous with AI. Now, this might sound facetious, but to help avert another AI winter (maybe 10 years down the road), we need to be loud and vocal, educating at the very least the investing strata of society as to hyponymy/hypernymy relationships between these terms.

It is bound to happen when researchers uses words like "Machine" and "Learning" to describe their field. Why not use words that actually describe it, like function approximation.

Re: The Limits of Machine Learning

#20

Post by joe_the_user »

pron wrote: post #17
wrong_variable wrote: post #4

How is this article on nautil.us ? Did the author just read the wikipedia.com page on Machine Learning ? There is an entire field in ML called unsupervised learning. Labelling data that do not have labels attached to them. Its not a Fundamental (uggh) limit, I am not sure if the author even knows what Fundamental means, its not like the halting problem, or heat death of the universe. ML is also a very young field wit…

1. Just because someone decided to use the words "intelligence" and "neural" when describing a class of statistical clustering algorithms often based on backpropagation of errors doesn't mean these algorithms have anything to do with the brain or intelligence, and if they do, the relationship is not necessarily direct and immediate. Speaking about the two as if the connection is clear only muddles our understanding.…

We're not sure what intelligence is, and it is easy to show that it is not "a general ability to solve problems". Humans are great at solving some problems and pretty terrible at others.

Now wait a second. I would say humans are better at solving some problems directly and computers programmed by humans are better at solving other problems.

However, a computer with a single, fixed program alone will choke completely at some problems and humans are far more robust at finding a solution or at least "dealing" with any problem whatsoever one throws at them.

In the end, you're right that we don't know what intelligence is. And so just about any description is going to be somewhat tautological but "general problem solving ability" seems relatively less tautological than other concepts - ie, "general problem solving ability" seems about right for some value of "general" which we'll have to determine as we go along.

Post reply on HN