Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AAAS: Machine learning 'causing science crisis'

bbc.co.uk

Post reply on HN
Print view View original

11–20 of 119 posts

Re: AAAS: Machine learning 'causing science crisis'

#11

Post by raverbashing »

stiff wrote: post #8

A dishonest scientist can mine a dataset for statistically significant hypotheses and for a long time no institutional protection against it was in place: https://en.wikipedia.org/wiki/Data_dredging https://www.xkcd.com/882/ Machine learning makes it easier to test great many hypothesis, but even going fully "by hand" it is very easy to deviate from what the statistical framework of hypothesis testing would demand. T…

As your number of samples increase the chance that a hidden variable that explains the phenomenon but correlates with the thing you're testing also increases.

All experiments have a limit it seems

Re: AAAS: Machine learning 'causing science crisis'

#12

Post by harry8 »

boomskats wrote: post #2

Good read. It's also refreshing to see a mainstream article that talks about ML without once mentioning 'AI'.

ML is statistics with a different name using a computer, that should always be mentioned in articles for the general public.

On the other hand AI is fantasy BS hype boosting off the fact that ML sounds similar to AI to people who aren't aware Machine Learning is stats.

Maybe AI one day but today it is utterly ridiculous. No really. Every single article should mention both of those things at least in passing. Downvote away all you AI hype Surfers but you know it's true.

Re: AAAS: Machine learning 'causing science crisis'

#13

Post by [deleted] »

fock wrote: post #7

An undergrad to his supervisor in our office talking about publishing a paper: I've fixed the data, now the plots look ok. I (undergrad too) am sitting there thinking - well, you are using ML as a regression blackbox to plot a line, I can do that too w/o ML if I'm fixing the data. Supervisor: ok, that's really great. Me cringing... I'm not hammering the ML-keyword above my work (and thus am getting considerably less…

[deleted]

Re: AAAS: Machine learning 'causing science crisis'

#14

Post by marmaduke »

harry8 wrote: post #6

Fails to touch on the perverse incentives in academia, "publish or perish" etc. Torturing a dataset to find a p value that a journal will like (or equivalent stat measure) is better for your career than not publishing a paper that will be discredited in time. You have no incentive at all to decide "my results are unconvincing at this point, I'm not going to submit them" and every reason to write them up as a useful c…

This is the unfortunate truth. Furthermore, those who take more time to find a general, robust and theoretically sound result (beyond just the p-optimized publishable result), get filtered out of te tenure positions as not have a high impact factor, despite their papers in fact having a higher impact.

Re: AAAS: Machine learning 'causing science crisis'

#17

Post by polskibus »

harry8 wrote: post #6

Fails to touch on the perverse incentives in academia, "publish or perish" etc. Torturing a dataset to find a p value that a journal will like (or equivalent stat measure) is better for your career than not publishing a paper that will be discredited in time. You have no incentive at all to decide "my results are unconvincing at this point, I'm not going to submit them" and every reason to write them up as a useful c…

Seems like we need general solutions to overfitting, not just in machine learning.

Re: AAAS: Machine learning 'causing science crisis'

#18

Post by avip »

marmaduke wrote: post #14
harry8 wrote: post #6

Fails to touch on the perverse incentives in academia, "publish or perish" etc. Torturing a dataset to find a p value that a journal will like (or equivalent stat measure) is better for your career than not publishing a paper that will be discredited in time. You have no incentive at all to decide "my results are unconvincing at this point, I'm not going to submit them" and every reason to write them up as a useful c…

This is the unfortunate truth. Furthermore, those who take more time to find a general, robust and theoretically sound result (beyond just the p-optimized publishable result), get filtered out of te tenure positions as not have a high impact factor, despite their papers in fact having a higher impact.

You may have meant "bullshitable result".

Re: AAAS: Machine learning 'causing science crisis'

#19

Post by sampo »

bayesian_horse wrote: post #15

The other day someone lamented that you can't get published as an honest ML researcher, because other scientists are rendering whole professions obsolete all the time...

> you can't get published as an honest ML researcher

If you research ML, you can publish in ML journals, there are several. If your research is about applying ML to domain problems, are you then an ML researcher or a domain researcher?

Re: AAAS: Machine learning 'causing science crisis'

#20

Post by konschubert »

harry8 wrote: post #6

Fails to touch on the perverse incentives in academia, "publish or perish" etc. Torturing a dataset to find a p value that a journal will like (or equivalent stat measure) is better for your career than not publishing a paper that will be discredited in time. You have no incentive at all to decide "my results are unconvincing at this point, I'm not going to submit them" and every reason to write them up as a useful c…

Some days ago there was a post here on HN about a post doc who failed to get tenure.

And every other comment was like: “What did he expect, he had much less than the usual two papers a year.”

So it seems that even here on HN, the mindset of quantity over quality still persists.

Post reply on HN