Live data from Hacker News

Ask HN: Is deep learning obsession in college ill founded?

news.ycombinator.com

11–20 of 37 posts

Re: Ask HN: Is deep learning obsession in college ill founded?

#11
> I understand the principles behind it, even find it cool, but the whole inscrutable nature of it is problematic to me.

Spot on.

On top of that we have 'AI' models getting fooled over adversarial attacks which just involve a single pixel change. As long as these issues are not tackled or not researched well enough, then we'll be pretty much be heading into another AI Winter and the hype cycle will go through its trough of disillusionment phase. Being unable to inspect the black-box nature of such deep-learning systems is why highly regulated industries involving danger to life such as healthcare or other safety critical industries label deep-learning solutions as unsafe for them.

Sure, all you see right now are other students and startups 'applying' deep learning everywhere, but they are hardly advancing the field unlike DeepMind and OpenAI are. In terms of learning, it's something good to learn as a student at college, but creating a AI startup now requires using Google, Amazon or Microsofts data center's for training which is clearly not sustainable anyway.

Security related projects and research are always where it's at.

Re: Ask HN: Is deep learning obsession in college ill founded?

#12

I'm an ML engineer, and I agree with you- deep learning is by far the most common approach for new problems in informatics. Imo deep learning is so popular because it "works". For a classification problem, if you try a linear baseline and a deep learning model, and you do a reasonable job of hyperparameter tuning and experimental design, it's likely you will outperform a simpler model. This holds true across many pro…

A lot of those comparisons strike me as shaky.

It's easy to beat a naive logistic regression model with a good neural network, but the gap often closes once you start trying to tune the logistic model too. (And it's not like the neural networks aren't tuned either--architecture search, data augmentation, etc).

Recent review on medical data: https://www.sciencedirect.com/science/article/abs/pii/S08954...

Re: Ask HN: Is deep learning obsession in college ill founded?

#13
post #4

Personally I think deep-learning is a bubble, and it will soon collapse to its natural place in computer science. Which is not to say that it's a fad that will disappear, only that it will retreat to being just a regular tool among the many tools we have for solving different kinds of problems. Its inscrutable nature is definitely problematic for some use-cases, and not so problematic for others.

I've been doing the data thing for a while. During one of my defenses of R, someone brought up that R was a black hole. That if you programmed in R, you were a user who just filled in the correct function arguments and it just spit out the answer. And that was when my thoughts on machine learning changed.

The vast majority of us are users. We massage the data to be in a certain shape, then feed it through a machine that someone else created. We can change the parameters. We can change the data. But few of us are going to look in to the code of a random forest function.

I've switched tracks and started doing web development. Playing with the hyper parameters in machine learning is no different than changing the feel of a drop down by changing the colors, fonts and other things to fit a certain aesthetic.

I could be wrong, but I have yet to meet anyone that has done anything besides use packages created by others to call themselves data scientists. I think that opens it up to becoming just another tool no different than Excel.

Re: Ask HN: Is deep learning obsession in college ill founded?

#14
post #11

> I understand the principles behind it, even find it cool, but the whole inscrutable nature of it is problematic to me. Spot on. On top of that we have 'AI' models getting fooled over adversarial attacks which just involve a single pixel change. As long as these issues are not tackled or not researched well enough, then we'll be pretty much be heading into another AI Winter and the hype cycle will go through its tro…

> As long as these issues are not tackled or not researched well enough, then we'll be pretty much be heading into another AI Winter and the hype cycle will go through its trough of disillusionment phase.

It's not all or nothing as you present it. ML models can be useful even if they are imperfect - and we should not forget humans aren't perfect either. For example, a model could reduce 50% of the time necessary to enter an invoice into the database. It's imperfect, yet useful.

A model need not run alone without any safety. It can have plain old programming rules to validate its outputs, or use human in the loop.

> Sure, all you see right now are other students and startups 'applying' deep learning everywhere, but they are hardly advancing the field unlike DeepMind and OpenAI are.

On the contrary, I would say that what DeepMind and OpenAI are doing is largely irrelevant for industry. There is a huge number of domains where no ML model has been created, and that is because there are so few people who can make them. The low hanging fruit hasn't been picked yet. It's like electricity at the beginning of the 20th century. The work these students and startups are doing is the good, useful work. You don't need DeepMind grade models to solve most real problems.

> creating a AI startup now requires using Google, Amazon or Microsofts data center's for training

You can train most useful models on a single machine today. Some, like Logistic Regression, train in seconds or minutes. Others take an hour, or a day. Some heavy ones take a week to train. If you don't do hyper-parameter search or cutting edge research you only need a few runs to get a working model. It's data tagging that usually takes months or years.

Re: Ask HN: Is deep learning obsession in college ill founded?

#15
post #4

Personally I think deep-learning is a bubble, and it will soon collapse to its natural place in computer science. Which is not to say that it's a fad that will disappear, only that it will retreat to being just a regular tool among the many tools we have for solving different kinds of problems. Its inscrutable nature is definitely problematic for some use-cases, and not so problematic for others.

While I agree with your sentiment regarding ML engineers - they are just another kind of devs, and that's where it will go - I think DL is not just a tool like any other from the software toolbox. It's more like a paradigm changer, like the print, the engine, electricity, communication and computing. It tends to eat the world.

Re: Ask HN: Is deep learning obsession in college ill founded?

#16

I'm an ML engineer, and I agree with you- deep learning is by far the most common approach for new problems in informatics. Imo deep learning is so popular because it "works". For a classification problem, if you try a linear baseline and a deep learning model, and you do a reasonable job of hyperparameter tuning and experimental design, it's likely you will outperform a simpler model. This holds true across many pro…

A lot of those comparisons strike me as shaky. It's easy to beat a naive logistic regression model with a good neural network, but the gap often closes once you start trying to tune the logistic model too. (And it's not like the neural networks aren't tuned either--architecture search, data augmentation, etc). Recent review on medical data: https://www.sciencedirect.com/science/article/abs/pii/S08954...

Logistic regression is exactly a NN with no hidden layers and a sigmoid activation function. A feedforward NN with additional layers is strictly more expressive than logistic regression.

Re: Ask HN: Is deep learning obsession in college ill founded?

#17
post #9

My situation is the same as yours, CS junior heading into my capstone project next semester, and my opinion is a resounding yes. The deep learning obsession is almost certainly a hype bubble. I have observed the same here at my university, the "But what if we did it with deep learning?" projects are almost reaching meme status. It's rather disheartening as someone who actually is interested in AGI, but I've been driv…

> How is the work in deep learning helping us understand the nature of intelligence

Neural networks performance on a problem is a benchmark of its real difficulty. It gives us insight, a new perspective.

In millennia of deliberations what have philosophers have discovered about the nature of intelligence? And then .. a neural net beats us at all board games, another can solve differential equations, another can translate, another can see, and so on. Have we really not learned anything by these inventions?

Another advantage of DL is that it frames the problem of intelligence in mathematical concepts and rigorous evaluation.

I, for one, have reconsidered all my spiritual beliefs after learning about the agent-environment-reward model of reinforcement learning. A new way of framing the agent and life, so parsimonious and powerful. And it does not require a soul, or a god, or anything outside out real environment, and yet can explain so much.

The whole machine learning paradigm is another powerful concept through which we can understand how we might function. Previously you might wonder how emotion, thought, sensation, imagination and will relate to each other. Now we can understand how they might be implemented and wired together, and what principles support their function.

Re: Ask HN: Is deep learning obsession in college ill founded?

#18
post #15
post #4

Personally I think deep-learning is a bubble, and it will soon collapse to its natural place in computer science. Which is not to say that it's a fad that will disappear, only that it will retreat to being just a regular tool among the many tools we have for solving different kinds of problems. Its inscrutable nature is definitely problematic for some use-cases, and not so problematic for others.

While I agree with your sentiment regarding ML engineers - they are just another kind of devs, and that's where it will go - I think DL is not just a tool like any other from the software toolbox. It's more like a paradigm changer, like the print, the engine, electricity, communication and computing. It tends to eat the world.

> It's more like a paradigm changer, like the print, the engine, electricity, communication and computing.

Either we really disagree about deep learning, or you vastly underestimate the influence of the other technologies that you've listed.

Re: Ask HN: Is deep learning obsession in college ill founded?

#20
22 years ago when I was in your shoes, distributed systems were the topic of the day, and all of us were going to be building systems with CORBA and DCOM... so guess what my project and paper were about? That's right, things I never touched in my career, but darn it if they didn't help me get my first job because they were hot topics of the moment.

So, pick something in "AI" that is the hotness of the moment, learn what you can, do your best, and then get on with life and career.

Post reply on HN