Possible projects that might keep you occupied for the entire year:
Make a Rust front-end for the GNU Compiler Collection.
Emulate something.
Write a hypervisor.
Ask HN: Is deep learning obsession in college ill founded?
21–30 of 37 posts
Re: Ask HN: Is deep learning obsession in college ill founded?
#22Personally I think deep-learning is a bubble, and it will soon collapse to its natural place in computer science. Which is not to say that it's a fad that will disappear, only that it will retreat to being just a regular tool among the many tools we have for solving different kinds of problems. Its inscrutable nature is definitely problematic for some use-cases, and not so problematic for others.
I've been doing the data thing for a while. During one of my defenses of R, someone brought up that R was a black hole. That if you programmed in R, you were a user who just filled in the correct function arguments and it just spit out the answer. And that was when my thoughts on machine learning changed. The vast majority of us are users. We massage the data to be in a certain shape, then feed it through a machine t…
years ago on the first ML hype wave I completed the excellent MOOC by Andrew Ng. In that course, he did go through the math and it was helpful to me to understand what was going on under the hood, but even then the value of ML wasn't understanding what it was doing, but understanding if your model was doing something well. I think your take that using packages created by others will be mostly what we do moving forward, and that's also true of pretty much all software development.
Re: Ask HN: Is deep learning obsession in college ill founded?
#23Earlier quoted context omitted.
A lot of those comparisons strike me as shaky. It's easy to beat a naive logistic regression model with a good neural network, but the gap often closes once you start trying to tune the logistic model too. (And it's not like the neural networks aren't tuned either--architecture search, data augmentation, etc). Recent review on medical data: https://www.sciencedirect.com/science/article/abs/pii/S08954...
Logistic regression is exactly a NN with no hidden layers and a sigmoid activation function. A feedforward NN with additional layers is strictly more expressive than logistic regression.
Yes! The million dollar question is how much of that expressivity is actually required.
In many papers, the "baseline" logistic regression model is very stripped down: y~logit(.) but the neural network has had its expressiveness optimized in various ways. People aren't comparing against a 3 layer feedfoward network; there's augmentation and pre-training, architecture search and special learning schemes.
My claim is that if you want to claim that a problem needs the expressivity that (only) a neural network provides, you ought to be devoting a great deal of effort to the logistic regression model too. Make it a steelman, rather than a strawman, if you will.
Re: Ask HN: Is deep learning obsession in college ill founded?
#24Personally I think deep-learning is a bubble, and it will soon collapse to its natural place in computer science. Which is not to say that it's a fad that will disappear, only that it will retreat to being just a regular tool among the many tools we have for solving different kinds of problems. Its inscrutable nature is definitely problematic for some use-cases, and not so problematic for others.
> Its inscrutable nature is definitely problematic for some use-cases, and not so problematic for others.
It's a problem whereever reliable operation is required, or analytic tractability (explanation) is required, or where resources available for data labeling are limited.
Its niche appears quite small, unless and until solid mathematical foundations are developed for it.
Re: Ask HN: Is deep learning obsession in college ill founded?
#25My situation is the same as yours, CS junior heading into my capstone project next semester, and my opinion is a resounding yes. The deep learning obsession is almost certainly a hype bubble. I have observed the same here at my university, the "But what if we did it with deep learning?" projects are almost reaching meme status. It's rather disheartening as someone who actually is interested in AGI, but I've been driv…
> How is the work in deep learning helping us understand the nature of intelligence Neural networks performance on a problem is a benchmark of its real difficulty. It gives us insight, a new perspective. In millennia of deliberations what have philosophers have discovered about the nature of intelligence? And then .. a neural net beats us at all board games, another can solve differential equations, another can trans…
> And then .. a neural net beats us at all board games, another can solve differential equations, another can translate, another can see, and so on. Have we really not learned anything by these inventions?
I would still argue no, we haven't learned anything about intelligence from these. They are impressive achievements, but strictly in the sense of "We found a way to use computers in a way we were not using them before".
1.) Neural nets + MCTS beat us at all board games--board games that humans invented and can achieve mastery at. If a human Go player was born who could beat that version of AlphaZero, we would not say that person has solved intelligence.
2.) Differential equations: also invented/discovered by humans, can also be solved by humans
3.) Translate and see: See above, with the additional caveat that humans actually are better at translating and seeing than deep learning systems are.
In addition, these were all achieved individually by systems with different architectures and massive amounts of training data that would amount to several human lifetimes. An 18 year-old human can play board games, drive a car, do differential equations, and learn multiple languages, with a single brain using a generalized structure and a fraction of the "training data" afforded to DL. This indicates to me that ML as a whole is still very far off the mark of General Intelligence.
> Previously you might wonder how emotion, thought, sensation, imagination and will relate to each other. Now we can understand how they might be implemented and wired together, and what principles support their function.
Previously? This is still an unanswered question. Show me where deep learning research has even come close to producing a system that can learn and adapt like a human mind does.
Re: Ask HN: Is deep learning obsession in college ill founded?
#26My situation is the same as yours, CS junior heading into my capstone project next semester, and my opinion is a resounding yes. The deep learning obsession is almost certainly a hype bubble. I have observed the same here at my university, the "But what if we did it with deep learning?" projects are almost reaching meme status. It's rather disheartening as someone who actually is interested in AGI, but I've been driv…
> How is the work in deep learning helping us understand the nature of intelligence Neural networks performance on a problem is a benchmark of its real difficulty. It gives us insight, a new perspective. In millennia of deliberations what have philosophers have discovered about the nature of intelligence? And then .. a neural net beats us at all board games, another can solve differential equations, another can trans…
>> Have we really not learned anything by these inventions?
Not really, and that is the problem. We can create something that sort of works - but we don't understand it. The only thing we have learned is that we don't need to understand intelligence to build something that works on some tasks.
Re: Ask HN: Is deep learning obsession in college ill founded?
#27Earlier quoted context omitted.
While I agree with your sentiment regarding ML engineers - they are just another kind of devs, and that's where it will go - I think DL is not just a tool like any other from the software toolbox. It's more like a paradigm changer, like the print, the engine, electricity, communication and computing. It tends to eat the world.
> It's more like a paradigm changer, like the print, the engine, electricity, communication and computing. Either we really disagree about deep learning, or you vastly underestimate the influence of the other technologies that you've listed.
I'd maybe buy it if you broadened the claim to "quantification" or something. It's undoubtedly true that aggressively collecting and analyzing data has transformed society a lot: Taylorism, mass production, bureaucracies, science (not just data), even Guinness. However, this has been going on for ~150 years already.
As for deep learning specifically...meh.
Re: Ask HN: Is deep learning obsession in college ill founded?
#28Possible projects that might keep you occupied for the entire year: Make a Rust front-end for the GNU Compiler Collection. Emulate something. Write a hypervisor.
Thanks, it helps. I'm actively looking for project ideas.
Re: Ask HN: Is deep learning obsession in college ill founded?
#29Personally I think deep-learning is a bubble, and it will soon collapse to its natural place in computer science. Which is not to say that it's a fad that will disappear, only that it will retreat to being just a regular tool among the many tools we have for solving different kinds of problems. Its inscrutable nature is definitely problematic for some use-cases, and not so problematic for others.
While I agree with your sentiment regarding ML engineers - they are just another kind of devs, and that's where it will go - I think DL is not just a tool like any other from the software toolbox. It's more like a paradigm changer, like the print, the engine, electricity, communication and computing. It tends to eat the world.
That's exactly what I'm arguing against: I think the "eating the world" part is a hype-cycle. I think DL has truly revolutionized a handful of very narrow cases - computer vision and speech recognition/synthesis, for example - but that people are vastly over-estimating how "paradigm-changing" it actually is.
Re: Ask HN: Is deep learning obsession in college ill founded?
#30Personally I think deep-learning is a bubble, and it will soon collapse to its natural place in computer science. Which is not to say that it's a fad that will disappear, only that it will retreat to being just a regular tool among the many tools we have for solving different kinds of problems. Its inscrutable nature is definitely problematic for some use-cases, and not so problematic for others.
> Its inscrutable nature is definitely problematic for some use-cases, and not so problematic for others. It's a problem whereever reliable operation is required, or analytic tractability (explanation) is required, or where resources available for data labeling are limited. Its niche appears quite small, unless and until solid mathematical foundations are developed for it.
It shines the most in "soft computing"; computer vision, etc. These also tend to be the least-important areas to explain or audit, partly just because they're so trivial to verify with nothing but human intuition.
Where it becomes problematic - and where DL isn't actually very well-suited anyway - is making "real decisions"; things that would normally be backed by rigid logic.