Are you sure that overall deep learning customers are spending more than traditional ML customers? My guess: it’s not even close - financial sector is still order of magnitude larger as paying customer than those hot CV/NLP companies.
Machine Learning: The Great Stagnation
161–170 of 227 posts
Re: Machine Learning: The Great Stagnation
#162This article is dead-on, but I think it is missing a fairly large segment of where ML is actually working well: anomaly detection and industrial defect detection. While I agree that everyone was shocked, myself included, when we saw how well SSD and YOLO worked, the last mile problem is stagnating. What I mean is: 7 years ago I wrote an image pipeline for a company using traditional AI methods. It was extremely chall…
>The industry is stagnated for exactly the reasons brought up: we don't know how to squeeze out the last mile problem because NNs are EFFING HARD and research is very math heavy: e.g., it cannot be hacked by a Zuck-type into a half-assed product overnight, it needs to be carefully researched for years. This makes programmers sad, because by nature we love to brute force trial-and error our code, and homey don't play that game with machine learning.
Uh what? You can literally finetune a Fast.ai model overnight to be borderline SOTA on whatever problem you have data for. 0 Math involved, isn't that exactly a hacker's wet dream?
Re: Machine Learning: The Great Stagnation
#163Lots of salty kids in here trying to justify spending 6 years as an anonymous hyperparameter tuning drone in the Stanford AI Lab.
Re: Machine Learning: The Great Stagnation
#164This article is dead-on, but I think it is missing a fairly large segment of where ML is actually working well: anomaly detection and industrial defect detection. While I agree that everyone was shocked, myself included, when we saw how well SSD and YOLO worked, the last mile problem is stagnating. What I mean is: 7 years ago I wrote an image pipeline for a company using traditional AI methods. It was extremely chall…
> However, places where it isn't stagnating are things like vibration and anomaly detection. This is a case where https://github.com/YumaKoizumi/ToyADMOS-dataset really shines because it adds something that didn't exist before, and it doesn't have to be 100% perfect: anything is better than nothing.
This is a link to a dataset, unless I'm missing something it's not about anomaly detection. I looked into this area a few years ago and always try to keep my eye open for breakthroughs... care to share any other links?
Re: Machine Learning: The Great Stagnation
#165Winter is coming, again.
Winter is always coming, the important question is when it will arrive. Current ML research is still destroying new problems with ease so the current velocity is high. It will slow down first, before people start to question why the new crop of problems are too hard, and then the cycle will start again.
I think there's already a good amount of discussion around where current ML methods. stuff like lack of sample efficiency, adjusting for distribution shift, etc.
Re: Machine Learning: The Great Stagnation
#166Earlier quoted context omitted.
I think you are exactly right, and want to add a couple things ( ): - It's easy to recognize scientific / technological revolutions in hindsight, but at the time they're anything but. I think most recognize the importance of persistence in the progenitor of an idea. What's often missed in these discussions is how important the subsequent incremental progress is: working out consequences of a theoretical insight, figu…
Where in the world did you come across the ( ) ... ( ) style? I like it.
Found the LISP programmer
Re: Machine Learning: The Great Stagnation
#167BERT engineer? That acronym is essentially ungooglable, since you just find every person ever named Bert. This is what I finally found: BERT, which stands for Bidirectional Encoder Representations from Transformers. https://arxiv.org/abs/1810.04805
Searching for "BERT machine learning" would have worked.
Re: Machine Learning: The Great Stagnation
#168Some good points in the article, but I disagree with the tone and the conclusion. > we’ve rewarded and lauded incremental researchers as innovators, increased their budgets so they can do even more incremental research There isn't a scientific field where every single paper is groundbreaking. It's a Brownian motion of small incremental innovations, until eventually we stumble upon something big (like deep learning).…
There isn't a scientific field where every single paper is groundbreaking. It's a Brownian motion of small incremental innovations, until eventually we stumble upon something big (like deep learning). In no way is machine learning unique in this.
Except ... while machine learning is great, has made important and significant strides, it's not a yet science. It involves essentially a series of sophisticated, mathematically informed recipes for feeding data to giant algorithms and having them create something useful (maybe very useful but still).
An analogy from a couple years ago is bridge building before physics. You accumulate rules of thumb, you get a vague understand what works. You get better. But you aren't producing a systematic field.
And that implies merely advancement isn't necessarily progress (which isn't to say there's no progress but building larger SOTA isn't that as the article notes).
Re: Machine Learning: The Great Stagnation
#169> It’s important to avoid becoming Gary Marcus and criticize existing technique that work without proposing something else that works even better. Hilarious. Has Gary Marcus actually done anything, in practical terms, like actual code or something, that outperforms the DL approaches he attacks so viciously?
I can't direct a movie but I can tell that Plan 9 from Outer Space is crap (or so crap it's gold, whatever). The kid who pointed out the emperor had no clothes didn't need to be a master tailor. And Gary Marcus doesn't need to propose alternatives to current deep learning practice in order for his criticism of deep learning to be valid.
... and yet he has: neuro-symbolic integration. That's his suggestion.
And in fact that's a whole field that's been publishing work for a while now. So he's not just making it up.
Re: Machine Learning: The Great Stagnation
#170Earlier quoted context omitted.
I think you are exactly right, and want to add a couple things ( ): - It's easy to recognize scientific / technological revolutions in hindsight, but at the time they're anything but. I think most recognize the importance of persistence in the progenitor of an idea. What's often missed in these discussions is how important the subsequent incremental progress is: working out consequences of a theoretical insight, figu…
Where in the world did you come across the ( ) ... ( ) style? I like it.
[deleted]