Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning: The Great Stagnation

marksaroufim.substack.com

Post reply on HN
Print view View original

141–150 of 227 posts

Re: Machine Learning: The Great Stagnation

#141

Post by b3kart »

p1esk wrote: post #136
AndrewKemendo wrote: post #80

Earlier quoted context omitted.

>We've been in an exciting deep learning craze for a while, but it's silly to expect it to last forever. Back to the grind now. This was effectively my response to hardmaru when this topic came up on reddit [1] Basically 2010-2018 was an open field for ML/DL research with old(ish) methods being rapidly applied to low hanging fruit and large datasets with newly cheap compute. Deepmind and others are actually making ne…

Those other approaches you mentioned don’t get much love because they don’t work. Their advocates worked on them for many years and have nothing to show for it.

>they don't work.

In the same way that ANNs didn't work for quite some time, until we've had the compute and the data to train them successfully?

I get that it's important to prove that an idea is worthwhile, and the easiest way to do that is to use it to solve a practical problem. At the same time, I am conscious that we shouldn't put all our eggs in the deep learning basket: who knows where the ceiling is going to be.

Don't get me wrong, I like deep learning, and you have to be silly not to admit how successful it has been. But the field would be so much more boring if not for the people with alternative views and ideas.

Re: Machine Learning: The Great Stagnation

#142

Post by dcow »

kkylin wrote: post #130
b3kart wrote: post #43

Some good points in the article, but I disagree with the tone and the conclusion. > we’ve rewarded and lauded incremental researchers as innovators, increased their budgets so they can do even more incremental research There isn't a scientific field where every single paper is groundbreaking. It's a Brownian motion of small incremental innovations, until eventually we stumble upon something big (like deep learning).…

I think you are exactly right, and want to add a couple things ( ): - It's easy to recognize scientific / technological revolutions in hindsight, but at the time they're anything but. I think most recognize the importance of persistence in the progenitor of an idea. What's often missed in these discussions is how important the subsequent incremental progress is: working out consequences of a theoretical insight, figu…

Where in the world did you come across the () ... () style? I like it.

Re: Machine Learning: The Great Stagnation

#143

Post by xpe »

gbolcer wrote: post #60
Ericson2314 wrote: post #25

I still feel like much of AI is a plot to dumb down the modern economy. We want our business people to be just as effective as our quants; we want nothing to require real intellectual labor. The idea that you traditionally have these programmers who spout mumbo-jumo all day, cost a lot of money, and seem to always be planning stuff behind your back is threatening, and all the more so because you are utterly dependent…

It's just another tool. I have this saying: all machine learning is just clever tricks and techniques until someone learns how to brute force the solution. It's the same adoption/business technology tension that has existed since Frederick Taylor in the early 1900s or Vonnegut's Player Piano concept where they propose taking a recorder to automated human-adverse tasks by recording their movements. The hype is trying…

> I have this saying: all machine learning is just clever tricks ...

It appears to me that you are missing the mark here, unless this is largely a definitional issue.

Do you consider the foundations of ML to be a clever trick?

Do you think human brains primarily learn by clever tricks?

When a metaphor or saying falls apart with one more level of questioning, I would suggest it may be time to find a better metaphor.

Re: Machine Learning: The Great Stagnation

#144

Post by gok »

SoSoRoCoCo wrote: post #115

This article is dead-on, but I think it is missing a fairly large segment of where ML is actually working well: anomaly detection and industrial defect detection. While I agree that everyone was shocked, myself included, when we saw how well SSD and YOLO worked, the last mile problem is stagnating. What I mean is: 7 years ago I wrote an image pipeline for a company using traditional AI methods. It was extremely chall…

> This makes programmers sad, because by nature we love to brute force trial-and error our code, and homey don't play that game with machine learning.

Huh? If anything I would say ML is way more trial-and-error focused than imperative programming.

Re: Machine Learning: The Great Stagnation

#145

Post by p1esk »

b3kart wrote: post #141
p1esk wrote: post #136

Earlier quoted context omitted.

Those other approaches you mentioned don’t get much love because they don’t work. Their advocates worked on them for many years and have nothing to show for it.

>they don't work. In the same way that ANNs didn't work for quite some time, until we've had the compute and the data to train them successfully? I get that it's important to prove that an idea is worthwhile, and the easiest way to do that is to use it to solve a practical problem. At the same time, I am conscious that we shouldn't put all our eggs in the deep learning basket: who knows where the ceiling is going to…

No, not in the same way. Convnets and lstms worked great when they were invented in the 90s. Yes, they were limited by the available compute power but they did work well right away.

Re: Machine Learning: The Great Stagnation

#146

Post by mloncode »

formalsystem wrote: post #131

Hey everyone this is OP, was a really nice surprise seeing my article generating so much discussion. I unfortunately don't think I'll have time to answer everyone but feel free to reply to this comment if you'd like to ask me anything. It seems like the article was a bit polarizing, some of the comments made me realize I made a few imprecise statements and I'll fix those. Other comments didn't approve of my tone and…

Great job writing this article.

Re: Machine Learning: The Great Stagnation

#147

Post by sanxiyn »

otabdeveloper4 wrote: post #118
rolisz wrote: post #53

Earlier quoted context omitted.

Image tagging is not in that category. Google Photos, Synology Moments and other similar projects are valuable for end users. Nobody had time to manually tag hundreds of gigabytes of photos with even 10% of the items in them.

Like I said, I'm not talking about end users. (Obviously for end users ML enables doing lots of things "for free" that used to be paid services before.) I'm talking about commercial applications, like the OP said. That is, things I could potentially pitch to management and substantiate with something concrete that isn't "you can fire your lowest-paid contractors now".

Some things are worth doing but not worth paying people. Google is using ML to read house numbers from Google Street View photos. Why is that not a good commercial application? Google gets its map updated faster and cheaper with ML.

Re: Machine Learning: The Great Stagnation

#148

Post by crypto420 »

otabdeveloper4 wrote: post #19
superbcarrot wrote: post #12

Earlier quoted context omitted.

That's unlikely. There are many machine learning applications that have been shown to be good enough for commercial use and they aren't going anywhere. The worst case for the field is that progress slows down, people realise that their expectations were unrealistic and the hype inevitably dies down. Which has to happen eventually. So even if ML isn't the hottest thing or a massively growing field, it will still be us…

> There are many machine learning applications that have been shown to be good enough for commercial use and they aren't going anywhere. If you could name three of them I'd be really grateful. Serious question; everything surrounding ML seems to be only good for (non-monetizable) art projects. As art it is amazing, not going to lie, but "commercial use" seems like a huge stretch.

Not sure what you're talking about. Neural networks are powering a LOT of the internet.

> Siri/Cortana/Google Home/Alexa, powered by DeepSpeech+language models

> Google Search, powered by BERT

> Tesla, powered by variants of YOLO

> Facial recognition powered by MTCNN+FaceNet

> AirBnB product search+recommendations

> Amazon product recommendations

GANs are a bit artsy, but JFC they're not even a decade old - we've gone from shitty MNIST clones to fully synthetic faces in the span of 5 years!

Predictions: 1. I suspect we're going to see DeepFakes in Hollywood - famous people might license their faces to movies that they might not have the time to star in

2. People are going to start building even more powerful versions of search, like combinations of CLIP

3. Neural networks still aren't optimized for edge devices - we're going to see a deluge of cheap drones with cutting edge computer vision by default

Re: Machine Learning: The Great Stagnation

#149

Post by IdiocyInAction »

otabdeveloper4 wrote: post #19
superbcarrot wrote: post #12

Earlier quoted context omitted.

That's unlikely. There are many machine learning applications that have been shown to be good enough for commercial use and they aren't going anywhere. The worst case for the field is that progress slows down, people realise that their expectations were unrealistic and the hype inevitably dies down. Which has to happen eventually. So even if ML isn't the hottest thing or a massively growing field, it will still be us…

> There are many machine learning applications that have been shown to be good enough for commercial use and they aren't going anywhere. If you could name three of them I'd be really grateful. Serious question; everything surrounding ML seems to be only good for (non-monetizable) art projects. As art it is amazing, not going to lie, but "commercial use" seems like a huge stretch.

Spam detection is the oldest and most well-known one.

It is also used quite a bit in graphics and imaging; DLSS is a consumer-facing application, but it is also used in other domains, like OCR.

Machine translation is another ubiquitous use case. As is any kind of language processing, like text-to-speech.

Also, in the industry, it is heavily used for anomaly and defect detection. Also, Google reportedly uses it for a lot of search/recommendation stuff.

ML is definitely monetizable, but not every company needs it, by a long shot. It is not "AI" and seems to often resemble a complex DSP step when used in practice. I think it's overhyped, but it is far, far more useful than anything blockchain will ever be.

Re: Machine Learning: The Great Stagnation

#150

Post by jpfr »

SilkStocking wrote: post #132
JPKab wrote: post #124

Earlier quoted context omitted.

Just to be clear, there's a lot of researchers making money that aren't producing squat. There are many AI ethicists I admire, and there are many that can't code, can't produce, and just spend their time getting into arguments with people like Yann LeCunn, who actually have produced groundbreaking research.

LeCon's work wasn't groundbreaking; he was saved from oblivion only by Moore's Law. LeCun stood still doing the same thing for decades. But when wave after wave of faster hardware arrived he was finally declared successful. So if you want to credit success to dumb luck, or to stubbornness, or insanity (as Einstein said: "doing the same thing over and over again and expecting different results." then LeCon is your man…

I hope calling him a “con” wasn’t intentional.

He speaks French after all...

Post reply on HN