Live data from Hacker News

AAAS: Machine learning 'causing science crisis'

bbc.co.uk

101–110 of 119 posts

Re: AAAS: Machine learning 'causing science crisis'

#101
The issue talked about here is distinct from the larger "reproducibility crisis"; the latter is a result of shoddily designed (or simply fraudulent) experimental work, whereas the issue here is the aggregate effects of the huge amount of computational work that is being done- even when that work is being done correctly and honestly.

Testing a hypothesis against a pre-existing dataset is a valid thing to do, and it is also almost trivially simple (and completely free) for someone with a reasonable computational background. There are researchers who spend a decent portion of their careers performing these analyses. This is all well and good- we want people to spend time analyzing the highly complex data that modern science produces- but we run into problems with statistics.

Suppose an analyst can test a hundred hypotheses per month (this is probably a low estimate.) Each analysis (simplifying slightly!) ends with a significance test, returning a p-value indicating the likelihood that the hypothesis is false. If p literally hundred-to-one. But you see the problem; even if we assume that this researcher tests no valid hypotheses at all over the course of a year, we would expect them to send out one paper per month- and each of these papers would be entirely valid, with no methodological flaws for reviewers to complain about.

In reality, of course, researchers sometimes test true hypotheses, and the rate of true to false computational-analysis papers would depend on the ratio of "true hypotheses that analysis successfully catches" to "false hypothesis that squeak by under the p-value threshold" (i.e., the True Positive rate vs the False Positive rate.) It's hard to guess that this ratio would be, but if AAAS is calling things a "crisis," it's clearly lower than we would like.

But there's a further problem, since the obvious solution- lower the p-value threshold for publication- would lower both the False Positive rate and the True Positive rate. The p-value that gets assigned to the results of an analysis of a true hypothesis are limited by the statistical power (essentially, size and quality) of the dataset being looked at; lower the p-value threshold too much, and analysts simply won't be able to make a sufficiently convincing case for any given true hypothesis. It's not a given that there is a p-value threshold for which the True Positive/False Positive ratio is much better than it is now.

"More data!" is the other commonly proposed solution, since we can safely lower the p-value threshold if we have the data to back up true hypotheses. But even if we can up the experimental throughput so much that we can produce True Positives at p < 0.0001, that simply means that computational researchers can explore more complicated hypotheses, until they're testing thousands or millions of hypotheses per month- and then we have the same problem. In a race between "bench work" and "human creativity plus computer science," I know which I'd bet on.

Re: AAAS: Machine learning 'causing science crisis'

#103
post #6

Fails to touch on the perverse incentives in academia, "publish or perish" etc. Torturing a dataset to find a p value that a journal will like (or equivalent stat measure) is better for your career than not publishing a paper that will be discredited in time. You have no incentive at all to decide "my results are unconvincing at this point, I'm not going to submit them" and every reason to write them up as a useful c…

Didn’t we used to value negative outcomes?

Re: AAAS: Machine learning 'causing science crisis'

#104
> Machine learning 'causing science crisis'

ML or more generally mathematics do not cause anything. People who misuse mathematics are to blame here. Some fields are simply using tools they don't understand and this predates ML advances by decades. Thinking of stats use in psychology and medicine for instance.

This trend of presenting ML are some kind of magic powder is ridiculous. I blame hyped presentations by influential ML scientists for this.

Re: AAAS: Machine learning 'causing science crisis'

#105
post #60

Earlier quoted context omitted.

Medicine may be better than ML but there’s not much in the difference. > COMPare: Qualitative analysis of researchers’ responses to critical correspondence on a cohort of 58 misreported trials > Background > Discrepancies between pre-specified and reported outcomes are an important and prevalent source of bias in clinical trials. COMPare (Centre for Evidence-Based Medicine Outcome Monitoring Project) monitored all tr…

I am just in the process of digging into this paper and covering it in an article, so I'm quite familiar with it. But as bad as this is: What the COMPare project is doing here is documenting the flaws of a process to counter bad scientific practice. The reality in most fields (including pretty much all of CS and ML) is that no such process exists at all, because noone even tries to fix these issues. So you have medic…

Are you planning on touching on autoML? Do you think that could help?

Re: AAAS: Machine learning 'causing science crisis'

#106

Science works because it posits models first, and then data is sought to confirm or disconfirm it. The benefit of having a model first is that it is much more likely to be general (and hence reproducible). ML does completely opposite. Data first, and then the model is discovered using data. It's pretty easy to see why it would lead to non-reproducible models.

From this perspective, where is the line between ML and automated p-hacking?

Re: AAAS: Machine learning 'causing science crisis'

#107
post #99
post #50

Earlier quoted context omitted.

The problem was there before, but there are reasons why Machine Learning is amplifying bad practices. In the past people were manually fishing for results in available datasets. Now they have algorithms to do it for them. In medicine a popular way to use ML is to improve diagnosis. Now there's already a problem in medicine that the benefits of early diagnosis are overrated and the downsides (overtreatment etc.) usual…

It seems what ML is really doing is exposing weaknesses in our scientific processes. The appropriate response here is to fix the processes, instead of blaming the latest fad and imploring people to "try harder". If the root cause isn't fixed, the next fad after ML will cause the same thing again. What feasible systemic changes can we make so that scientists can't get away with publishing sloppy results? It's not an i…

I think it is a very interesting question for a lot of scientists, but also an extremely hard one to answer. And an even harder one to implement.

Even obvious wins that almost everyone can agree on, like getting publishing out of the hands of for-profit entities that add no value is taking forever, because cultural, social and political institutions are hard to move.

Re: AAAS: Machine learning 'causing science crisis'

#108
post #40

I can see there being issues with reproducibility, i.e. getting the exact same results, but has there ever been a time when science was more replicable? Data/techniques/findings/papers are under more scrutiny than ever. No positive results will be taken as sacrosanct in CS anymore. This is a complete 180 from 10+ years ago.

In my view, rather than talking about time periods, it might be preferable to consider different fields of science wrt replicability. In fact it doesn't even make sense to put all of "science" in one basket. The medical and behavioral sciences get the most attention these days, but are not comparable to physics, chemistry, geology, astronomy, etc. My field (physics) doubtlessly has its own problems, yet has produced theories of astounding generality and accuracy in spite of potential flaws in the individual studies that led to the success of those theories.

Replication might not turn out to be the big problem. The lack of progress towards a unifying theory might be a more important long term issue.

Re: AAAS: Machine learning 'causing science crisis'

#109
post #44
post #36

Earlier quoted context omitted.

What method would you use to yield scientific knowledge in the sense of cause-reaction? Many important processes really do have large numbers of causal factors that interact non-linearly. If we want to try to learn about that, some statistical method that deals with many parameters will be needed. Such models are generally referred to as "Machine Learning". Their generalisation or causal inference properties are part…

A tool called Mathematics which can exactly describe this interactions. And if those processes have a lot of variables, a ML-model might certainly be useful, but it will never be generally applicable! This probably also contributes to "scientific knowledge" but it's not the same as scientific facts (or whatever you call universally transferable results).

You're building your mathematical model based on the knowledge you have, which is from the data you have, and there is still the same risk that your theory won't generalize to new observations.

Re: AAAS: Machine learning 'causing science crisis'

#110

Earlier quoted context omitted.

It's possible you're right, but that's not what people were saying IIRC.

That's exactly what they were saying. Go read the thread again. They were being prescriptive of academic careers, not descriptive of the author's inherent merit as a researcher.

and to recurse a layer, i’m being descriptive of the prescriptivists that i know to exist and i easily believe did that guy in.
Post reply on HN