I believe neural networks are over hyped sometimes. They are not always the best tool for the job. There are lots of other ML techniques such as SVM, naive Bayes, k-nearest neighbor, decision tree, logistic regression, random forest etc. nobody is using because they lack the hype factor. If something lacks some keywords like neural network, deep learning, reinforced learning, than it is deemed not cool.
I can't think of anything that neural nets can't beat, except small tabular data with boosted decision trees. Can you give some examples?
> we often use ML over DL in scientific analysis because we need models that can be inspected/explained not just results
> also, DL generally requires more data whereas you can get by with ML on less data if you have domain knowledge