Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning: The Great Stagnation

marksaroufim.substack.com

Post reply on HN
Print view View original

111–120 of 227 posts

Re: Machine Learning: The Great Stagnation

#111

Post by minimaxir »

Ericson2314 wrote: post #25

I still feel like much of AI is a plot to dumb down the modern economy. We want our business people to be just as effective as our quants; we want nothing to require real intellectual labor. The idea that you traditionally have these programmers who spout mumbo-jumo all day, cost a lot of money, and seem to always be planning stuff behind your back is threatening, and all the more so because you are utterly dependent…

As someone who works on open-source tooling for AI, good.

The more people who can leverage AI, the better the industry will be as a whole. It serve as both a check on AI hype, and also leads innovating in encouraging new applications.

Huggingface Transformers is a good demo of this philosophy in action.

Re: Machine Learning: The Great Stagnation

#112

Post by dkislyuk »

otabdeveloper4 wrote: post #19
superbcarrot wrote: post #12

Earlier quoted context omitted.

That's unlikely. There are many machine learning applications that have been shown to be good enough for commercial use and they aren't going anywhere. The worst case for the field is that progress slows down, people realise that their expectations were unrealistic and the hype inevitably dies down. Which has to happen eventually. So even if ML isn't the hottest thing or a massively growing field, it will still be us…

> There are many machine learning applications that have been shown to be good enough for commercial use and they aren't going anywhere. If you could name three of them I'd be really grateful. Serious question; everything surrounding ML seems to be only good for (non-monetizable) art projects. As art it is amazing, not going to lie, but "commercial use" seems like a huge stretch.

Any online service with considerable content creation and recommendation that you can think of is using ML to improve every aspect of the system (better embeddings, better retrieval, better ranking, better text understanding, better personalization, etc.) And the commercial usefulness is measurable.

Industrial robotics, driver assist, drug discovery, computational photography, speech translation, and so many other examples illustrate a clear commercial applicability of ML methods scaled in the last 10 years specifically.

Re: Machine Learning: The Great Stagnation

#113

Post by DaedPsyker »

I've talked to colleagues about this before but I find it's mostly a cyclical nature fuelled by tunnel vision.

Every good couple of years researchers come up with a good advancement or fresh concept that reignites the community. However all that happens till the next breakthrough is basic tweaks. The amount of junk papers that slightly adjust the method that gets a few decimal place improvement then call it a snazzy name would fill a mountain. Drawing blood from a stone. People get so obsessed with specific methods thinking its the new great thing they don't stop and think it's probably not the only way.

Marketing and media is the worst though for general public perception. The amount of times they would warp ML into a magical pangea. "ML will make you skinny!" I mean it's just mathematical approximations, been around a while.

Don't get me wrong, I like this field. I'm fortunate that I get to apply it to a problem that helps people but at times I just want to shout from the rooftops that it's not a God it wont make all your dreams come true. Then they get annoyed it doesn't and we a bunch of stagnation articles like this.

Re: Machine Learning: The Great Stagnation

#114

Post by zetazzed »

Astronomy has entered a period of great stagnation. More and more grad students are investing huge amount of time and building larger and larger radio telescopes just to learn more about black hole formation and the properties of pulsars. Little consideration is given to how to more efficiently use inexpensive consumer telescopes purchased at Walmart! Where are the big discoveries? New planets in our solar system? A new structure for the Milky Way? I sincerely hope that the Julia language will allow us to discover ET radio signals, because we all know that Python lacks this capability.

Re: Machine Learning: The Great Stagnation

#115

Post by SoSoRoCoCo »

This article is dead-on, but I think it is missing a fairly large segment of where ML is actually working well: anomaly detection and industrial defect detection.

While I agree that everyone was shocked, myself included, when we saw how well SSD and YOLO worked, the last mile problem is stagnating. What I mean is: 7 years ago I wrote an image pipeline for a company using traditional AI methods. It was extremely challenging. When we saw SSDMobileNet do the same job 10x faster with a fraction of the code, our jaws dropped. Which is why the dev ship turned on a dime: there's something big in there.

The industry is stagnated for exactly the reasons brought up: we don't know how to squeeze out the last mile problem because NNs are EFFING HARD and research is very math heavy: e.g., it cannot be hacked by a Zuck-type into a half-assed product overnight, it needs to be carefully researched for years. This makes programmers sad, because by nature we love to brute force trial-and error our code, and homey don't play that game with machine learning.

However, places where it isn't stagnating are things like vibration and anomaly detection. This is a case where https://github.com/YumaKoizumi/ToyADMOS-dataset really shines because it adds something that didn't exist before, and it doesn't have to be 100% perfect: anything is better than nothing.

At Embedded World last year I saw tons of FPGA solutions for rejecting parts on assembly lines. Since every object appears nearly in canonical form (good lighting, centered, homogeneous presentation), NN's are kicking ass bigtime in that space.

It is important to remember Self-Driving Car Magic is just the consumer-facing hype machine. ML/NNs are working spectacularly well in some domains.

Re: Machine Learning: The Great Stagnation

#116

Post by [deleted] »

rowanG077 wrote: post #14
jarym wrote: post #4

> Academics sacrifice material opportunity costs in exchange for intellectual freedom. Most academics I’ve come across only think they’re doing this. My perception is they are too insecure about their self-worth to pursue material opportunities. I admit, the number of academic types I know is not vast so maybe it’s too small a subset to make any judgments

The core problem is that you don't have intellectual freedom. You won't get funded if you are not researching the hot new thing.

[deleted]

Re: Machine Learning: The Great Stagnation

#118

Post by otabdeveloper4 »

rolisz wrote: post #53
otabdeveloper4 wrote: post #41

Earlier quoted context omitted.

Okay, I should have worded my comment more carefully. These applications seem to firmly fall into the "I'm willing to compromise on quality if I don't have to pay a living person a wage" niche, so they're value-destroying, not value-creating. Are there examples of value-creating applications for ML? (From a business point of view; obviously the "shitty translations but at no cost" proposition creates value for the av…

Image tagging is not in that category. Google Photos, Synology Moments and other similar projects are valuable for end users. Nobody had time to manually tag hundreds of gigabytes of photos with even 10% of the items in them.

Like I said, I'm not talking about end users. (Obviously for end users ML enables doing lots of things "for free" that used to be paid services before.)

I'm talking about commercial applications, like the OP said. That is, things I could potentially pitch to management and substantiate with something concrete that isn't "you can fire your lowest-paid contractors now".

Re: Machine Learning: The Great Stagnation

#119

Post by otabdeveloper4 »

bidirectional wrote: post #74
otabdeveloper4 wrote: post #19

Earlier quoted context omitted.

> There are many machine learning applications that have been shown to be good enough for commercial use and they aren't going anywhere. If you could name three of them I'd be really grateful. Serious question; everything surrounding ML seems to be only good for (non-monetizable) art projects. As art it is amazing, not going to lie, but "commercial use" seems like a huge stretch.

As far as I'm aware, ML is driving a pretty sizeable amount of new functionality at the big tech companies. How do you think Google does search and translate, Netflix recommends films, Tesla drives your car, Siri recognises your voice, etc. etc. I see the sentiment you've expressed a lot, and feel it speaks to a massive disconnect amongst developers. Most people are interacting with ML systems dozens if not hundreds…

Question wasn't if ML is useful. The question is - does ML make money or create (financial) value for the company?

The stuff you listed doesn't, it's just part of a moat for already established products that don't depend on ML for their market share. (It's not like Netflix will lose market share if they switch from ML to some other approach for their recommendation system.)

Re: Machine Learning: The Great Stagnation

#120

Post by dumb1224 »

bendergarcia wrote: post #75
funcDropShadow wrote: post #38

Earlier quoted context omitted.

> Now, that's not to say I am against labor saving devices. That's good to know. Because "labor saving devices" is by no means the field of machine learning. A hammer is a "labor saving device" if all you have are rocks. I've got the impression that a lot of people conflate machine learning and robotics with labor saving in general. Of course compared to the state of art machine learning let's us hope to find magical…

Agreed I find that ml is a meta tool. You gotta Know the tool you need, then use ml to build it. Build the wrong tool and there’s little value. The right tool will make people more productive.

I always see that the data used in ML field are not being praised enough. I don't see enough people realising it was the large collection of data / curated information that enabled the advancement (may it be that the ML methods were heavily and manually tweaked to accomplish certain tasks). It simplify wasn't possible before the age of internet.
Post reply on HN