Title makes it sound as if the AAAS made this statement, its a single researcher who is making this claim.
AAAS: Machine learning 'causing science crisis'
31–40 of 119 posts
Re: AAAS: Machine learning 'causing science crisis'
#32An undergrad to his supervisor in our office talking about publishing a paper: I've fixed the data, now the plots look ok. I (undergrad too) am sitting there thinking - well, you are using ML as a regression blackbox to plot a line, I can do that too w/o ML if I'm fixing the data. Supervisor: ok, that's really great. Me cringing... I'm not hammering the ML-keyword above my work (and thus am getting considerably less…
Yup. As someone who is in a junior position in the field - I'm torn between, on one hand, riding the wave so to speak and taking advantage of all the buzzwords that I can put on my resume (which is fine with me because I can back them up), and on the other hand avoiding association with a certain type of person/career path that might turn out be just hot air in a couple of years. So if I can actually write decent cod…
Having been subject to a lot of buzzword-blarers over the last term I'd say that you just tell everybody that you are working the foundations of what all the other people are doing if they ask, what makes you special. If you can back it up, why not ride the wave. I would refrain from putting the sticker on everything though. I als made a turnaround and now spin everything I do which takes time (basically automating my research and data-vis) as building blocks for (gradschool) ML-based research.
Re: AAAS: Machine learning 'causing science crisis'
#33An undergrad to his supervisor in our office talking about publishing a paper: I've fixed the data, now the plots look ok. I (undergrad too) am sitting there thinking - well, you are using ML as a regression blackbox to plot a line, I can do that too w/o ML if I'm fixing the data. Supervisor: ok, that's really great. Me cringing... I'm not hammering the ML-keyword above my work (and thus am getting considerably less…
That supervisor is someone who "made it in academia" so it might be good to not sneer and cringe at them and your peers.
Re: AAAS: Machine learning 'causing science crisis'
#34Case in point : LHC Higgs results - how many detection's vs how many events? How were the detection's determined... The answer is with a large booster [1]
I postulate that out of 12 billion random events it would be remarkable if a booster didn't extract 100 or so items that looked similar to a Higgs detection.
Well, let's give it 20 years and a new generation of PI's who aren't invested in this and have grad students who are keen to find something different in the data.
But ohh.. all the data has been thrown aways... oh! [2]
[1] https://indico.cern.ch/event/705941/contributions/2897000/at...
[2] https://www.forbes.com/sites/startswithabang/2018/09/13/has-...
Re: AAAS: Machine learning 'causing science crisis'
#35Fails to touch on the perverse incentives in academia, "publish or perish" etc. Torturing a dataset to find a p value that a journal will like (or equivalent stat measure) is better for your career than not publishing a paper that will be discredited in time. You have no incentive at all to decide "my results are unconvincing at this point, I'm not going to submit them" and every reason to write them up as a useful c…
We need to find resources to fund "Failures in Science" journals that exclusively seek to publish interesting research that went nowhere. Personally I'd find these far more interesting to study. "Here's some background. Here's a pretty logical, plausible hypothesis we came up with and how, here's our experiment, here's our results, here's our thoughts as to why we were wildly wrong."
Instead of allows failures (which would be also good btw), i would increase the time at disposal needed to pubblish something worthwhile.
The greatest thinkers in history had no pressure outside their own urgence to solve/explore. Surely, not everyone is Galileo but today science need that kind of freedom.
Re: AAAS: Machine learning 'causing science crisis'
#36Curious (possibly naive) question: isn't there a fundamental difference between the goals behind creating models with ML vs the "old-fashioned" way? That is, in modern ML applications, you're creating a model with dozens/hundreds of potential variables, without a hypothesis of how they relate or contribute to the target (other than that they might, hence your including them in the modeling process). You're using the…
I think you got the point. A lot of people don't seem to realize that ML might be great for finding patterns but will never yield scientific knowledge in the sense of cause-reaction sense. Unfortunately everyone thinks he can use it for finding "new stuff" and so in my field they "predict material properties", etc. using ML fed with data where every review about the physics tells you that the algorithms they use for…
What method would you use to yield scientific knowledge in the sense of cause-reaction? Many important processes really do have large numbers of causal factors that interact non-linearly. If we want to try to learn about that, some statistical method that deals with many parameters will be needed. Such models are generally referred to as "Machine Learning". Their generalisation or causal inference properties are particular to each implementation and identification strategy, but you can't just say "ML will never yield scientific knowledge".
Re: AAAS: Machine learning 'causing science crisis'
#37An undergrad to his supervisor in our office talking about publishing a paper: I've fixed the data, now the plots look ok. I (undergrad too) am sitting there thinking - well, you are using ML as a regression blackbox to plot a line, I can do that too w/o ML if I'm fixing the data. Supervisor: ok, that's really great. Me cringing... I'm not hammering the ML-keyword above my work (and thus am getting considerably less…
regression is not a black box.
Re: AAAS: Machine learning 'causing science crisis'
#38Case in point : LHC Higgs results - how many detection's vs how many events? How were the detection's determined... The answer is with a large booster [1] I postulate that out of 12 billion random events it would be remarkable if a booster didn't extract 100 or so items that looked similar to a Higgs detection. Well, let's give it 20 years and a new generation of PI's who aren't invested in this and have grad student…
" all the data has been thrown aways... oh"
That's a ridiculously disingenuous summary of the article, did you read past the Forbes headline?
Re: AAAS: Machine learning 'causing science crisis'
#39Overfitting is a well-known problem in the ML community. There are methods to avoid this: cross validation, train-test splits, etc. There are also models that give you an estimate of the standard deviation of a prediction. What is the point? We don't need new algorithms, we just have to apply existing methods properly.
Re: AAAS: Machine learning 'causing science crisis'
#40I can see there being issues with reproducibility, i.e. getting the exact same results, but has there ever been a time when science was more replicable? Data/techniques/findings/papers are under more scrutiny than ever. No positive results will be taken as sacrosanct in CS anymore. This is a complete 180 from 10+ years ago.