Earlier quoted context omitted.
Technically yes, most often it's about stacking more layers in neural networks, making them "deep". However, there is some merit to the new hype since stacking more layers worked way better than anyone previously working with neural networks and ML thought it would. But in theory you could generalize deep learning to other methods than neural networks, it's basically about creating way more complex models than those…
> it's basically about creating way more complex models than those used in previous research and feeding them lots of data Those are instructions for over-fitting. Deep learning neural networks escape from this problem somehow, but it's not a given that other models would escape it too.
This is true! Overfitting is definitely one of the biggest problems with deep learning. Some techniques to avoid it have been developed, such as dropout (introducing noise) and early stopping. But in general this is why deep learning requires huge amount of data, a deep learning model will overfit if not given enough data. That is also why (at this time) it only performs well for certain problems where the ratio between available data and problem complexity is high enough.