for the moment I thought it was from 2003
A Survey of Deep Learning for Scientific Discovery
11–17 of 17 posts
Re: A Survey of Deep Learning for Scientific Discovery
#12In a survey on "scientific discovery", I would have expected more examples than face and image recognition and natural language processing, which are so stale at this point. Healthcare? Physics? Chemistry? Biology? Sociology?
This is just a bad title. Should have been named simply as “A Survey of Deep Learning”. This paper is an excellent and up to date overview of deep learning models, methods and best practices.
Re: A Survey of Deep Learning for Scientific Discovery
#13Always wonder who these kinds of reviews / surveys are for? Nobody is going to learn machine learning by reading a 50 page pdf. Meanwhile, people that have experience will have a hard time finding the info they don't already know. Opinionated & narrow >> Shallow & comprehensive
I will read it, to defend my non-DeepLearning choices for supervised ML .. so many on the bandwagon for unsupervised CNN with their GPUs
Re: A Survey of Deep Learning for Scientific Discovery
#14In a survey on "scientific discovery", I would have expected more examples than face and image recognition and natural language processing, which are so stale at this point. Healthcare? Physics? Chemistry? Biology? Sociology?
The rest you listed require inference and causality.
Deep learning does not do this.
Data with less noises are what most deep learning and non statistical models does well. Meaning that image, nlp, etc.. deep learning does well. But data with lots noises/uncertainty/variance or even data that isn't large enough, such as time series, currently statistical models are still king (https://en.wikipedia.org/wiki/Makridakis_Competitions).
Even with healthcare you're answering a question/ hypothesis. This is where statistical models strength lies because all statistical models are hypothesis tests and vice versa. There are very little opportunity in healthcare where you would use deep learning compare to statistic. I've seen NLP can be of use but the majority of work in healthcare are inference/casuality base (this is why they use propensity model so much). I'm in this space public healthcare.
Re: A Survey of Deep Learning for Scientific Discovery
#15In a survey on "scientific discovery", I would have expected more examples than face and image recognition and natural language processing, which are so stale at this point. Healthcare? Physics? Chemistry? Biology? Sociology?
The rest you listed require inference and causality. Deep learning does not do this. Data with less noises are what most deep learning and non statistical models does well. Meaning that image, nlp, etc.. deep learning does well. But data with lots noises/uncertainty/variance or even data that isn't large enough, such as time series, currently statistical models are still king ( https://en.wikipedia.org/wiki/Makridaki…
Interesting you mentioned Makridakis competitions. There's one going on right now on Kaggle, and the current leader believes a NN will be the winning model:
https://www.kaggle.com/c/m5-forecasting-accuracy/discussion/...
More generally, it seems that time series forecasting so far has mostly attracted statisticians with little DL experience [1]. Now that there is $50k prize, this will be a good test of whether statistical methods are "still king". If I were to enter this field, I'd probably look into latest transformer based models, especially the ones used to model raw audio data, e.g. [2].
There's also a real possibility that whenever any strong forecasting method is developed (DL based or otherwise) it's not published as the developers simply use it to make money (betting, stock market, etc).
[1] https://journals.plos.org/plosone/article?id=10.1371/journal...
Re: A Survey of Deep Learning for Scientific Discovery
#16Always wonder who these kinds of reviews / surveys are for? Nobody is going to learn machine learning by reading a 50 page pdf. Meanwhile, people that have experience will have a hard time finding the info they don't already know. Opinionated & narrow >> Shallow & comprehensive
I will read it, to defend my non-DeepLearning choices for supervised ML .. so many on the bandwagon for unsupervised CNN with their GPUs
I am misunderstood here.. it means, for the purposes that are appropriate, use a disciplined, supervised model.. and know the strengths and weakness' of the CNN models.. yes, some reaction to the hype of CNN..
Re: A Survey of Deep Learning for Scientific Discovery
#17Earlier quoted context omitted.
The rest you listed require inference and causality. Deep learning does not do this. Data with less noises are what most deep learning and non statistical models does well. Meaning that image, nlp, etc.. deep learning does well. But data with lots noises/uncertainty/variance or even data that isn't large enough, such as time series, currently statistical models are still king ( https://en.wikipedia.org/wiki/Makridaki…
Interesting you mentioned Makridakis competitions. There's one going on right now on Kaggle, and the current leader believes a NN will be the winning model: https://www.kaggle.com/c/m5-forecasting-accuracy/discussion/... More generally, it seems that time series forecasting so far has mostly attracted statisticians with little DL experience [1]. Now that there is $50k prize, this will be a good test of whether statis…
I'll wait to see the result at the end of the competition.
This is just one of the two competitions for m5. The other one is uncertainty.