Live data from Hacker News

Past Performance is Not Indicative of Future Results (2020)

locusmag.com

181–190 of 285 posts

Re: Past Performance is Not Indicative of Future Results (2020)

#181
post #176

We are paying for the incredible bamboozle that is the phrase "Machine Learning." If we used computerized statistical inference instead and the phrase "machine learning" did not exist the attitude to people from investors to regulators, from customers, vendors, doom sayers and boosters alike would be vastly better taken as whole. Nearly everyone here knows mostly when seeing AI written or hearing it that it's a total…

"AI" as a term deserves this rep, because it was effectively marketing as far back as the 70s. But you're way off on "Machine Learning".

There's been plenty of progress in the last 15 years re-interpreting many ML methods as regression (any optimization is a regression if you set up the right likelihood function). But many important results and techniques -- including today's ubiquitous deep nets -- originated and had successful applications way before they had statistical interpretations. They came from fields like compression theory, database design, or even biological interpretations.

The term Machine Learning was introduced to re-focus the field on a measurable objective: algorithms that improve with more data. The "Learning" part was not an abstract term to tug on your imagination, but included formal definitions of how algorithms improve that involved slightly fewer assumptions than statistical learning (which is a subfield).

This lineage isn't that important today, but that focus on how learning is measured is still the most important guidepost both for ML research and for sorting out marketing BS from realistic claims. Certainly, state of the art work using deep nets for tasks like NLP, image and video recognition aren't designed by reasoning about the statistical interpretation, or tested by applying typical statistical tests. Popularizing this work as Statistical Inference or Regression wouldn't give any added intuition and wouldn't really describe the way ML research proceeds, or how ML systems succeed or fail.

Re: Past Performance is Not Indicative of Future Results (2020)

#182
post #148
post #74

Earlier quoted context omitted.

> I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ field that leads to a general AI > I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. This isn't how science works though. Quoting the wikipedia page for Thomas Kuhn's "The Structure of Scientific Revolutions" ( https://en.w…

I think you're misunderstanding Kuhn slightly. He invented the term paradigm shift. What he means by normal science with intertwined spurts of revolution is more provocative. He means that in order to observe periods of revolution, the "dogma" of normal science must be cast aside and new normal must move in to replace it. Normal science hits a wall, gets stuck in a "rut" as Kuhn describes it. I think, in a way, Docto…

This is an excellent post. Thank you!

I felt like the part that wasn't in line with Kuhn was the idea that there was something wrong with a field if incremental improvement couldn't lead to a breakthrough like AGI. You're right. He's arguing Kuhn's point. But he seems to use it to conclude that machine learning is a dead end when it comes to AGI. Further, he seems to think this means AGI won't happen any time soon.

But, if I'm not misinterpreting Kuhn again, knowing that a revolution is necessary to overturn the current dogma (which I would argue is deep learning) doesn't tell us anything about when the revolution will occur. It could be tomorrow or 50 years from now or never. So, specifically, it doesn't tell us anything about machine learning in general, whether AGI is possible, or when AGI will happen.

Re: Past Performance is Not Indicative of Future Results (2020)

#183

Earlier quoted context omitted.

Thanks for your reply. I suppose a quick way to summarize my criticism is that it reads to me like you've dismissed the strengths of ML on technical grounds, while you imply you don't have any real technical experience in the field. You make a superficial comparison between the compexity of biology and ML, without providing any real insight, just saying one has lots going on and the other is matrix multiplication. If…

You don’t have to be an expert in a field to recognize that the current popular approaches to something aren’t even close to getting there.

Thanks for your insight!

Re: Past Performance is Not Indicative of Future Results (2020)

#184
post #176

We are paying for the incredible bamboozle that is the phrase "Machine Learning." If we used computerized statistical inference instead and the phrase "machine learning" did not exist the attitude to people from investors to regulators, from customers, vendors, doom sayers and boosters alike would be vastly better taken as whole. Nearly everyone here knows mostly when seeing AI written or hearing it that it's a total…

"AI" as a term deserves this rep, because it was effectively marketing as far back as the 70s. But you're way off on "Machine Learning". There's been plenty of progress in the last 15 years re-interpreting many ML methods as regression (any optimization is a regression if you set up the right likelihood function). But many important results and techniques -- including today's ubiquitous deep nets -- originated and ha…

It works by fitting curves. Whether you have a (presumably mathematical) "Statistical interpretation" or not is basically irrelevant in terms of what it actually does and what we should be conveying to people who aren't knowledgeable of the field. This is not about an academic argument.

Putting stats right there in the name is vastly, vastly more informative than "Learning" which has the nuance for 99% of people as something requiring intelligence and is misleading. Hence the AI cons all pop up immediately there are some public ML wins called "Learning."

Generalizing from data is actually what statistics does. It's what ML is. People like Hinton, Wasserman, Tibrishani et al seem to agree that ML is statistics but even that isn't what I'm talking about here.

Re: Past Performance is Not Indicative of Future Results (2020)

#185
post #176

We are paying for the incredible bamboozle that is the phrase "Machine Learning." If we used computerized statistical inference instead and the phrase "machine learning" did not exist the attitude to people from investors to regulators, from customers, vendors, doom sayers and boosters alike would be vastly better taken as whole. Nearly everyone here knows mostly when seeing AI written or hearing it that it's a total…

I see these dismissals as "it's just statistics" often, and I don't get where they come from. If anything maybe it's "just" stochastic gradient descent, but there is a distinct "learning" pareto ML that does not obviously follow out of statistics. You could argue it's just addition, subtraction, multiplication, division and root extraction too, but that is a pointless reduction that doesnt help understand what's going on.

Re: Past Performance is Not Indicative of Future Results (2020)

#186
post #143

Earlier quoted context omitted.

Idk maybe it's semantics, inference to me sounds more like a logical leap is happening, whereas in my mind the simplest form of inductive reasoning is just expecting a pattern to repeat itself.

Expecting a pattern to repeat itself may not be sufficient to count as intelligence, but general purpose pattern recognition certainly seems to fit the bill.

Computer Aided Pattern Recognition sounds reasonable in setting public expectations.

Re: Past Performance is Not Indicative of Future Results (2020)

#187
post #109

Earlier quoted context omitted.

Yeah I agree - during undergrad, I spent a few years studying neuroscience, and I was very let down by my first ML/AI course. Compared to what I had learned about the brain, what we called an "ANN" just seemed like such a silly toy. The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epig…

I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/a…

The quirky thing to remember about gpt-3 is that it really is just a giant autocomplete based on the internet. It can do math insofar as it’s memorized some text which did that math with slightly different verbiage etc.

If you ask it to compute something that would never have been seen on the internet it’s likely to fail. E.g. add 2 extremely large/rare numbers together

Re: Past Performance is Not Indicative of Future Results (2020)

#188
post #153

Earlier quoted context omitted.

I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/a…

> It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? Few have predicted a reasonably-capable text-writing engine or automatic video face replacement, but many have predicted self-driving cars would have been re…

Automatic video face replacement always seemed an obvious application to me. I'm more surprised that the tools for it are still so rough. I guess we can thank social taboos for that.

When I was a kid, I remember wondering how Soviets were obsessed with faking photos. A few years later, I saw Terminator 2 and realized that faking videos was also a thing. The tools for it would clearly get better and better over time. When I studied ML in the early 2000s, it seemed obvious that pattern recognition tasks such as image manipulation would be "easy" for computers, once we found the right approach and made the ML systems big enough. In the end, I decided not to pursue ML, because jobs were still scarce and I found discrete problems more interesting. That was probably the worst career mistake I've ever made.

Re: Past Performance is Not Indicative of Future Results (2020)

#189
post #153

Earlier quoted context omitted.

I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/a…

> It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? Few have predicted a reasonably-capable text-writing engine or automatic video face replacement, but many have predicted self-driving cars would have been re…

People tend to predict simple technological substitutes for human tasks rather than novel things. I suspect we won't get artificial humans because we'll end up not actually wanting that and getting something better instead. Just like we got cars instead of artificial horses.

Re: Past Performance is Not Indicative of Future Results (2020)

#190

Earlier quoted context omitted.

A parameter in a neural network is more comparable to a synapse of which the brain has 100 trillion. And yes, we will get there too one day.

That's a good point, but amount of training data these neural networks take doesn't seem compatible to me. If I read all day everyday from the moment I was born until now, I couldn't have read 45 terabytes of text.

Don't forget the training that you did before you were born through evolution. It wasn't text but was a bunch of transferrable skills that help us understand text.
Post reply on HN