Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientific datasets are riddled with copy-paste errors

sciencedetective.org

Post reply on HN
Print view View original

51–60 of 68 posts

Re: Scientific datasets are riddled with copy-paste errors

#51

Post by mattkrause »

jcattle wrote: post #30

Just a thought: This data engineering can only really occur in sciences with a significant "moat". Expensive tools, expensive test setups, live, gene-altered animals, etc. In fields such as deep learning or other more digital fields (my field is using a lot of freely available satellite data) replication is often cheaper and actual application of research outcomes is a lot more common.

I used to think that but....

I've reviewed for a few "replication tracks" at ML Conferences and there are a surprising number of reports where people are simply unable to replicate published results. The reasons are all over the map: sometimes the original authors' code just needs to be fixed (new libraries, different environments), but other results simply don't seem to hold up.

Re: Scientific datasets are riddled with copy-paste errors

#52

Post by db48x »

IanCal wrote: post #34
jcattle wrote: post #29

Earlier quoted context omitted.

Yes, immediately thought the same. CSV alone is a footgun and a half on any computer which doesn't have . as the decimal separator. Let alone column sorting and joining of data.

CSV occupies, even years after moving away from more raw data work, way too much of my brain is still dedicated to "ways of dealing with CSV from random places". I can already hear people who like CSV coming in now, so to get some of my bottled up anger about CSV out and to forestall the responses I've seen before * It's not standardised * Yes I know you found an RFC from long after many generators and parsers were w…

> "You can edit it in a text editor" which feels like a monkeys-paw wish

Yes :) Although I will note that some editors are good enough to maintain the structure as the user edits. Consider Emacs with `csv-mode`, for example. Of course most users don’t have Emacs so they’ll just end up using notepad (or worse, Word).

Re: Scientific datasets are riddled with copy-paste errors

#53

Post by abstractdev »

IanCal wrote: post #34
jcattle wrote: post #29

Earlier quoted context omitted.

Yes, immediately thought the same. CSV alone is a footgun and a half on any computer which doesn't have . as the decimal separator. Let alone column sorting and joining of data.

CSV occupies, even years after moving away from more raw data work, way too much of my brain is still dedicated to "ways of dealing with CSV from random places". I can already hear people who like CSV coming in now, so to get some of my bottled up anger about CSV out and to forestall the responses I've seen before * It's not standardised * Yes I know you found an RFC from long after many generators and parsers were w…

[flagged]

Re: Scientific datasets are riddled with copy-paste errors

#55

Post by mcdonje »

jcattle wrote: post #44
mcdonje wrote: post #42

Earlier quoted context omitted.

You're blaming a lot of normal ETL problems on DSVs. Like, specifying date as a type for a field in JSON isn't going to ensure that people format it correctly and uniformly. You still have parsing issues, except now you're duplicating the ignored schema for every data point. The benefit you get for all of that overhead is more useful for network issues than ensuring a file is well formed before sending it. The people…

I would kind of disagree. We are talking here in the context of scientific datasets. Of course ETL plays a part here. However here it is really more the interplay of Excel with CSV which is often outputted by scientific instruments or scientific assistants. You get your raw sensor data as a csv, just want to take a look in excel, it understandably mangles the data in attempt to infer column types, because of course i…

That's Excel's type inference causing problems. Not an issue with CSV or any other type of DSV.

It is possible to import a CSV into Excel without type conversion. I just tested it two different ways.

While possible, it's not Excel's default way of doing things. Not always obvious or easy. Not enough people who use Excel really know how to use it.

Regardless, Excel mangling files via type inference is an Excel problem. It's not the fault of the file formats Excel reads in.

Re: Scientific datasets are riddled with copy-paste errors

#56

Post by thunderfork »

mcdonje wrote: post #55
jcattle wrote: post #44

Earlier quoted context omitted.

I would kind of disagree. We are talking here in the context of scientific datasets. Of course ETL plays a part here. However here it is really more the interplay of Excel with CSV which is often outputted by scientific instruments or scientific assistants. You get your raw sensor data as a csv, just want to take a look in excel, it understandably mangles the data in attempt to infer column types, because of course i…

That's Excel's type inference causing problems. Not an issue with CSV or any other type of DSV. It is possible to import a CSV into Excel without type conversion. I just tested it two different ways. While possible, it's not Excel's default way of doing things. Not always obvious or easy. Not enough people who use Excel really know how to use it. Regardless, Excel mangling files via type inference is an Excel problem…

The file format being ambiguous and underspecified enough to mangle is, though.

Re: Scientific datasets are riddled with copy-paste errors

#57

Post by BobaFloutist »

isolli wrote: post #27
scotty79 wrote: post #24

Earlier quoted context omitted.

> The people who get caught red handed like this are lazy, incompetent and stupid. Being a cheat significantly correlates with laziness, incompetence and stupidity so there are probably very few cheats smart and diligent enough to not get caught.

The sample of cheaters that we know about is biased towards cheaters who get caught.

To those, yes, but also to the weirdly high number of cheaters that can't help bragging to everyone that'll listen about their cheating.

Re: Scientific datasets are riddled with copy-paste errors

#58

Post by mmooss »

Note this exchange with the OP author:

> [Paper author:] Englund's claim that the Model 680 "records raw light measurements as it sees them without any post-processing" is incorrect. [...] It converts analog intensity signals to absorbance values using Beer's law, rounding results to the nearest 0.001 OD.

> [OP author:] Ok, that was incorrect on my part and shows my lack of knowledge about photometers. I was attempting to paraphrase an email from Bio-rad where they said: “The system [Bio-rad 680] was very basic and recorded OD as seen, it did not do any onboard manipulation of the data, it gave raw results for the user to interpret.”

For someone focusing on accuracy and sloppy work, that's a significant problem. How much of the OP is based on their "lack of knowledge" and reckless application of ignorance?

For the first issue, the section titled "Verdict" is followed by,

> the authors have so far not responded.

I would not issue verdicts without making sure I understand the other's perspective (note that it is a requirement in courts of law). That applies especially when I lack direct knowledge or expertise.

When I haven't taken that step, I've learned a thousand times that my certainty usually reflects my lack of imagination, knowledge, or careful thought. Even when I'm 'right', I'm 'wrong': even if the 'verdict' is the same, the truth differs from what I was so certain about. I used to express my certainty prematurely; now I know to keep it to myself until I know what I'm talking about, which frequently saves me from major and/or embarassing errors.

Re: Scientific datasets are riddled with copy-paste errors

#59

Post by jcattle »

mcdonje wrote: post #55
jcattle wrote: post #44

Earlier quoted context omitted.

I would kind of disagree. We are talking here in the context of scientific datasets. Of course ETL plays a part here. However here it is really more the interplay of Excel with CSV which is often outputted by scientific instruments or scientific assistants. You get your raw sensor data as a csv, just want to take a look in excel, it understandably mangles the data in attempt to infer column types, because of course i…

That's Excel's type inference causing problems. Not an issue with CSV or any other type of DSV. It is possible to import a CSV into Excel without type conversion. I just tested it two different ways. While possible, it's not Excel's default way of doing things. Not always obvious or easy. Not enough people who use Excel really know how to use it. Regardless, Excel mangling files via type inference is an Excel problem…

If you get an .xls which doesn't have very esoteric functions, I expect it to open about the same way in any Excel program and any other office suite.

With CSV I do not have that expectation. I know that for some random user-submitted CSVs, I will have to fiddle. Even if that means finding the one row in thousand rows which has some null value placeholder, messing up the whole automatic inference.

Re: Scientific datasets are riddled with copy-paste errors

#60

Post by theshrike79 »

TrackerFF wrote: post #28

It wouldn't surprise me one bit if many of these things can be attributed to Excel usage. I'm a "power user" of excel, and when working on larger problems with tens of sheets, smaller mistakes can easily carry on. Even more so if you're not a proficient user. One of my first jobs as an analyst was to clean up messy spreadsheets made by people, even very senior employees, who never bothered to learn excel properly.

Excel doesn't have unit testing or validation built in.

That's my biggest problem in the world right now. SO MANY BIG THINGS in the world are running on random Excel sheets created by FSM knows who full of formulas and shit that have zero validation.

...and people are worried about "vibe coded slop" - at least that stuff is made with actual programming languages with unit testing frameworks.

Nobody has ever went through an inherited Excel sheet can confirmed that every field in column CB has the same formula and no-one in the 42 people long inheritance chain has accidentally fat-fingered a static number in there.

Post reply on HN