Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning: The Great Stagnation

marksaroufim.substack.com

Post reply on HN
Print view View original

91–100 of 227 posts

Re: Machine Learning: The Great Stagnation

#91

Post by visarga »

Ericson2314 wrote: post #25

I still feel like much of AI is a plot to dumb down the modern economy. We want our business people to be just as effective as our quants; we want nothing to require real intellectual labor. The idea that you traditionally have these programmers who spout mumbo-jumo all day, cost a lot of money, and seem to always be planning stuff behind your back is threatening, and all the more so because you are utterly dependent…

Wrong sub-field. Business insights are usually the domain of statistics, not deep learning. Deep learning is mostly for processing raw text, images, sounds and recommendations. It's dumb, boring stuff like translation, OCR, spam filtering and search ranking. How to run a business is the domain of game theory in economics, or maybe you could do something with Bayesian methods.

Re: Machine Learning: The Great Stagnation

#92

Post by MaxBarraclough »

b3kart wrote: post #43

Some good points in the article, but I disagree with the tone and the conclusion. > we’ve rewarded and lauded incremental researchers as innovators, increased their budgets so they can do even more incremental research There isn't a scientific field where every single paper is groundbreaking. It's a Brownian motion of small incremental innovations, until eventually we stumble upon something big (like deep learning).…

See also the recent discussion on Peer-reviewed papers are getting increasingly boring.

https://news.ycombinator.com/item?id=25605156

Re: Machine Learning: The Great Stagnation

#93

Post by CyberRabbi »

Interesting article but the part where he says “all” you need to learn are matrix multiplications seems vacuously true to the point of meaninglessness. All numerical methods across every field essentially boil down to matrix multiplications.

Your math ability needs to be such that you can transform a problem into a form that can be computed numerically using matrix multiplications, which requires more skill (sometimes significantly more) than simply knowing how matrix multiplications work. Sometimes this ability to reframe complex problems numerically using matrix multiplications is the lions share of the research!

Re: Machine Learning: The Great Stagnation

#94

Post by spongepoc »

screye wrote: post #32

There is about to be a 'great pivot' in ML. There has been a rabid frenzy of throwing money at anything that has ML in it. Soon investors and CEOs will realize that ML is effective in narrow ways and that not everything needs ML. They will also realize that 1 ML team + ML as a service (Azure ML, Sagemaker, Google AI platform) is cheaper and works more reliably. The services will keep improving and an underpaid medioc…

I think this analysis is completely spot on. Personally, I have had the same thoughts for about a year and I have just switched from a 'pure' ML/DS' to exactly the hybrid engineering style role you describe.

Re: Machine Learning: The Great Stagnation

#95

Post by adamc »

So, this is the claim I thought was interesting:

"Every paper is SOTA, has strong theoretical guarantees, an intuitive explanation, is interpretable and fair but almost none are mutually consistent with each other"

How can that be true unless the "theory" itself isn't really worked out?

Re: Machine Learning: The Great Stagnation

#96

Post by b3kart »

CyberRabbi wrote: post #93

Interesting article but the part where he says “all” you need to learn are matrix multiplications seems vacuously true to the point of meaninglessness. All numerical methods across every field essentially boil down to matrix multiplications. Your math ability needs to be such that you can transform a problem into a form that can be computed numerically using matrix multiplications, which requires more skill (sometime…

Good point. Almost as useful as saying "all you need to learn is Boolean algebra" when it comes to computing.

Re: Machine Learning: The Great Stagnation

#97

Post by z77dj3kl »

CyberRabbi wrote: post #93

Interesting article but the part where he says “all” you need to learn are matrix multiplications seems vacuously true to the point of meaninglessness. All numerical methods across every field essentially boil down to matrix multiplications. Your math ability needs to be such that you can transform a problem into a form that can be computed numerically using matrix multiplications, which requires more skill (sometime…

All you need is ZFC...

Re: Machine Learning: The Great Stagnation

#98

Post by counters »

AndrewKemendo wrote: post #80
b3kart wrote: post #43

Some good points in the article, but I disagree with the tone and the conclusion. > we’ve rewarded and lauded incremental researchers as innovators, increased their budgets so they can do even more incremental research There isn't a scientific field where every single paper is groundbreaking. It's a Brownian motion of small incremental innovations, until eventually we stumble upon something big (like deep learning).…

>We've been in an exciting deep learning craze for a while, but it's silly to expect it to last forever. Back to the grind now. This was effectively my response to hardmaru when this topic came up on reddit [1] Basically 2010-2018 was an open field for ML/DL research with old(ish) methods being rapidly applied to low hanging fruit and large datasets with newly cheap compute. Deepmind and others are actually making ne…

> Basically 2010-2018 was an open field for ML/DL research with old(ish) methods being rapidly applied to low hanging fruit and large datasets with newly cheap compute.

Absolutely! And what many folks need to continue to remember is that many scientific disciplines and domains are really just starting to wrestle with the utility and implications of this first generation of deep learning tools and applications. I graduated with my PhD in atmospheric science from an R1 just over 4 years ago; at that time, very few people were looking at how DL provided useful tools for their work. These days, the field is inundated with folks playing with these tools and knocking tons of low-hanging fruit off the tree - it might not be "deep", revolutionary research, but it's fomenting a mini-revolution with respect to R2O and real applications of what had previously been somewhat niche science.

There's no reason to think this trend won't continue. New tools let new generations of scientists take new stabs at their discipline, and of course the low-hanging fruit drops first as folks get their bearing, build skills/experience, and - most importantly - prove efficacy so that they can get funding for more ambitious work.

Re: Machine Learning: The Great Stagnation

#99

Post by jjk166 »

api wrote: post #56
Ericson2314 wrote: post #25

I still feel like much of AI is a plot to dumb down the modern economy. We want our business people to be just as effective as our quants; we want nothing to require real intellectual labor. The idea that you traditionally have these programmers who spout mumbo-jumo all day, cost a lot of money, and seem to always be planning stuff behind your back is threatening, and all the more so because you are utterly dependent…

Trying to use ML to get rid of programmers will just replace them with ML experts who also have to be programmers to implement the models and munge all the data. These people will in turn have to be paid more than the original programmers were.

The goal of ML isn't so much to get rid of programmers as to get rid of specialized programmers. Right now if you want to solve a problem, you need someone with a deep understanding of that specific problem to develop a solution. For a complex problem like diagnosing cancer patients, you are talking about a team of people with decades if not centuries of combined experience in oncology on top of the actual programming expertise to implement the tool. The holy grail of ML is to reach the point where someone who is an expert on making ML systems can apply the same (or substantially similar) tools and expertise to a wide range of problems - the same team that makes a cancer diagnosis system could also make a legal text search system or a protein folding system. Realistically we'll probably never get to the point where zero domain knowledge is required, but even if the requirement is just substantially reduced to the point where the ML expert can learn what they need in months instead of years, that would be revolutionary.

Re: Machine Learning: The Great Stagnation

#100

Post by kfk »

I fear ML and AI are taking the lion share of analytics resources in most big companies and that's just not right. We still have basic problems with data integrity and blending. Getting a real end to end view of any business process is still a herculean task. Try matching something like invoices with marketing leads, good luck...
Post reply on HN