Ask HN: Is deep learning obsession in college ill founded?

Background:

I'm a CS junior (about to become a senior) and in our last year, people choose their capstone project that they work on for the entire year. For some years (say since 2017), deep learning projects completely dominate the other projects in term of number and the awards that they go on to win. I understand the principles behind it, even find it cool, but the whole inscrutable nature of it is problematic to me.

I'm an ML engineer, and I agree with you- deep learning is by far the most common approach for new problems in informatics.

Imo deep learning is so popular because it "works". For a classification problem, if you try a linear baseline and a deep learning model, and you do a reasonable job of hyperparameter tuning and experimental design, it's likely you will outperform a simpler model. This holds true across many problem spaces.

I think the issue is that modern DL frameworks make it a little too easy to get pretty good performance on new problems. Other techniques generally require more background knowledge to make reasonable modeling assumptions, and still frequently perform worse than a naively applied DL approach.

I think DL will remain, in practice and education, a very popular tool. But it is essential to learn traditional statistical inference and other background to appropriately contextualize DL models so it isn't just some form of black magic.

It's a well funded research area and in many cases outperforms other solutions, so it's not going anywhere in Unis for the next decade at least.

It's not completely inscrutable in why it works, follow the thread on deep learning theory research by starting from the names here: http://www.vision.jhu.edu/tutorials/CVPR17-Tutorial-Math-Dee...

Deep learning is what is cool in CS right now. It lets you do new things that you couldn't do before. Based on that, it's going to be over-represented in projects by undergrads looking to do "cool" projects and show off their new-found skills.

But that isn't really a problem. In most cases, the projects you do as an undergrad don't affect your professional life in any way after you get your first job. Very, very few undergrad projects turn into real projects that anyone uses after the student graduates.

So don't worry too much about it. Ten years ago, every senior project was an app. Twenty years ago, every senior project was a website. It's just a sign of the times and doesn't matter in the long run.

My situation is the same as yours, CS junior heading into my capstone project next semester, and my opinion is a resounding yes. The deep learning obsession is almost certainly a hype bubble. I have observed the same here at my university, the "But what if we did it with deep learning?" projects are almost reaching meme status. It's rather disheartening as someone who actually is interested in AGI, but I've been driven away from wanting to pursue the field since the current research seems lacking in ambition and substance. My previous summer internship had me reading a lot of deep learning papers on arXiv, and the vast majority of them seem to be tweaking a single parameter in a DNN, achieving a 0.3% increase in score on an arbitrary benchmark, and calling it a meaningful result. I'd personally like to see more people doing work like the kind DeepMind does that seems to actually achieve breakthroughs informed by knowledge from neuroscience, but I have a feeling we won't see that anytime soon since DeepMind gets their pick of the best researchers in the world. I'm just an undergrad though, would love to hear the opinions of more knowledgeable people! Specifically, I'd like to hear arguments against the sentiment that "Deep learning right now is pretty much alchemy". How is the work in deep learning helping us understand the nature of intelligence, rather than just helping Facebook and Amazon better target advertisements and product recommendations?