Live data from Hacker News

Scientific datasets are riddled with copy-paste errors

sciencedetective.org

21–30 of 68 posts

Re: Scientific datasets are riddled with copy-paste errors

#21
What should give people pause is how not complicated (I'd hesitate to say easy) it would be to create a python script that would generate fake data such that it would be all but impossible to determine whether it's real or not. You just need to model the measuring device and hypothesis you want to support, then sample away.

The people who get caught red handed like this are lazy, incompetent and stupid. Makes you wonder what about the ones not getting caught.

Re: Scientific datasets are riddled with copy-paste errors

#22

This is legitimately so challenging to avoid, because loads of scientific processes are—to some degrees or others—bespoke and difficult to fully streamline and introduce efficient, well-structured, comprehensive QA. A LOT of labour goes into making it work. Most scientists I know and work with are very diligent people who care a lot about the outputs being as correct as possible, but wow, their workflows aren't great…

> their workflows aren't great Sounds like a startup idea.

If you want to make no money, sure.

The solutions these scientists need are bespoke and share little in common. They also have fixed grant funding.

In 2009 I made $15/hr working with some PhDs and grad students in a couple different labs to automate their workflows - I was the highest paid person in the room most of the time.

Re: Scientific datasets are riddled with copy-paste errors

#24

What should give people pause is how not complicated (I'd hesitate to say easy) it would be to create a python script that would generate fake data such that it would be all but impossible to determine whether it's real or not. You just need to model the measuring device and hypothesis you want to support, then sample away. The people who get caught red handed like this are lazy, incompetent and stupid. Makes you won…

> The people who get caught red handed like this are lazy, incompetent and stupid.

Being a cheat significantly correlates with laziness, incompetence and stupidity so there are probably very few cheats smart and diligent enough to not get caught.

Re: Scientific datasets are riddled with copy-paste errors

#25

What should give people pause is how not complicated (I'd hesitate to say easy) it would be to create a python script that would generate fake data such that it would be all but impossible to determine whether it's real or not. You just need to model the measuring device and hypothesis you want to support, then sample away. The people who get caught red handed like this are lazy, incompetent and stupid. Makes you won…

> The people who get caught red handed like this are lazy, incompetent and stupid. Being a cheat significantly correlates with laziness, incompetence and stupidity so there are probably very few cheats smart and diligent enough to not get caught.

Based on my experience of the field, I very much challenge that assumption.

Re: Scientific datasets are riddled with copy-paste errors

#26

Earlier quoted context omitted.

> The people who get caught red handed like this are lazy, incompetent and stupid. Being a cheat significantly correlates with laziness, incompetence and stupidity so there are probably very few cheats smart and diligent enough to not get caught.

Based on my experience of the field, I very much challenge that assumption.

Indeed, the niches of smart cheats or smart criminals have a lot of room. Because the trajectories to reach that stage without being caught by a legal (good legal work) or moral (good person) attractor are sparse and that makes them somewhat rare.

Handwaving correlations between cheating/criminality and most personality/intelligence aspects is an error, not least because there is a selection bias problem (eg. who gets caught).

Re: Scientific datasets are riddled with copy-paste errors

#27

What should give people pause is how not complicated (I'd hesitate to say easy) it would be to create a python script that would generate fake data such that it would be all but impossible to determine whether it's real or not. You just need to model the measuring device and hypothesis you want to support, then sample away. The people who get caught red handed like this are lazy, incompetent and stupid. Makes you won…

> The people who get caught red handed like this are lazy, incompetent and stupid. Being a cheat significantly correlates with laziness, incompetence and stupidity so there are probably very few cheats smart and diligent enough to not get caught.

The sample of cheaters that we know about is biased towards cheaters who get caught.

Re: Scientific datasets are riddled with copy-paste errors

#28
It wouldn't surprise me one bit if many of these things can be attributed to Excel usage. I'm a "power user" of excel, and when working on larger problems with tens of sheets, smaller mistakes can easily carry on. Even more so if you're not a proficient user.

One of my first jobs as an analyst was to clean up messy spreadsheets made by people, even very senior employees, who never bothered to learn excel properly.

Re: Scientific datasets are riddled with copy-paste errors

#29

It wouldn't surprise me one bit if many of these things can be attributed to Excel usage. I'm a "power user" of excel, and when working on larger problems with tens of sheets, smaller mistakes can easily carry on. Even more so if you're not a proficient user. One of my first jobs as an analyst was to clean up messy spreadsheets made by people, even very senior employees, who never bothered to learn excel properly.

Yes, immediately thought the same. CSV alone is a footgun and a half on any computer which doesn't have . as the decimal separator.

Let alone column sorting and joining of data.

Re: Scientific datasets are riddled with copy-paste errors

#30
Just a thought: This data engineering can only really occur in sciences with a significant "moat".

Expensive tools, expensive test setups, live, gene-altered animals, etc.

In fields such as deep learning or other more digital fields (my field is using a lot of freely available satellite data) replication is often cheaper and actual application of research outcomes is a lot more common.

Post reply on HN