Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autocorrect errors in Excel still creating genomics headache

nature.com

Post reply on HN
Print view View original

31–40 of 108 posts

Re: Autocorrect errors in Excel still creating genomics headache

#31

Post by prionassembly »

planet-and-halo wrote: post #11

Serious question, is this a viable area for a startup product? Obviously nothing will easily displace Excel for general spreadsheet work, but I could see room for a domain-specific spreadsheet that has core features and supports just a basic set of functions relevant to genomics, + a "we won't fuck up your data" feature.

The easier solution is coming up with a 1:1 human-readable-hashing scheme so that MARCH4 translates to "funny-blue-smell-tuesday". Then you keep the safe key in a column next to the unsafe key.

Re: Autocorrect errors in Excel still creating genomics headache

#32

Post by wtallis »

codetrotter wrote: post #28

That’s the price we pay. A reasonable alternative would be for example statically typing each column or row in the sheet. So in one sheet column A contains only floats, col B contains only text, col C contains only dates etc. And in another sheet col A may be date type, col B date type also, col C and D text, and col E monetary amounts. But this would come at the cost of not being able to mix types in column, and a l…

> But this would come at the cost of not being able to mix types in column, and a lot of people want to be able to mix.

I feel like a lot of the desire for this comes from Excel's desire to fill the screen with a single infinite spreadsheet. Apple's Numbers and some other programs make it more natural to have multiple separate tables on screen, each of which can have their own header rows and columns and separate type and format rules for those columns/rows. Excel more or less forces people to emulate this capability by just using a separate region of the same table, which gets in the way of applying consistent formatting and typing rules to whole columns or rows.

Re: Autocorrect errors in Excel still creating genomics headache

#33

Post by [deleted] »

guitarbill wrote: post #26

It's easy to hate on Excel or Microsoft, a "dumb" program and a faceless mega corporation. Don't get me wrong, I'm no fan. But the real question is what are journals doing? It seems like the obvious solution is for editors and reviewers to do a quick search for these errors. To me, this is more of an indication of how poor most journals really are. (You could argue that supplementary data should be in an open format,…

[deleted]

Re: Autocorrect errors in Excel still creating genomics headache

#34

Post by pletnes »

qayxc wrote: post #24
fartcannon wrote: post #16

Earlier quoted context omitted.

Yeah. At some point though, people who want to do something that requires python or R, but refuse to learn it, and then use Excel (and make mistakes like the article mentions) should reconsider their choices. This is really out of character for me, but in this instance, relying on excel to do everything is a mistake by the user, not Microsoft's ridiculous data destroying import function. Use the right tool.

> Use the right tool. Step one should be learn your tool, no matter the tool. There's several ways to avoid the issue in Excel. If someone working with Excel isn't able to learn that (heck, a simple template would suffice), I have no hope the same demographic would have any success with R or Python.

As someone who has programmed in python for a decade, and no idea how to do anything right in Excel, I would beg to differ.

Re: Autocorrect errors in Excel still creating genomics headache

#35

Post by dspillett »

cm2187 wrote: post #18
planet-and-halo wrote: post #11

Serious question, is this a viable area for a startup product? Obviously nothing will easily displace Excel for general spreadsheet work, but I could see room for a domain-specific spreadsheet that has core features and supports just a basic set of functions relevant to genomics, + a "we won't fuck up your data" feature.

That’s an expensive workaround to just formatting your cells as text before entering the values

You'd be surprised how much effort can be saved by not relying on human interaction to be reliable.

Though convincing people to switch to a new app for that reason, even if it were free and Free, would still be an uphill struggle.

> just formatting your cells as text before entering the values

It is a little more than that. I work with finance people, and there a lot of data is manipulated in Excel but passed around as CSV files for compatibility elsewhere. This causes no end of problems because fixes by setting cell properties are obviously lost in transcription, and date errors creep in as things are moved back & forth between people in the US and those in locales that do dates properly.

Re: Autocorrect errors in Excel still creating genomics headache

#36

Post by jasode »

OptionX wrote: post #23
an_d_rew wrote: post #4

Earlier quoted context omitted.

Because an absolutely enormous number of people involved in all ends of genomics and biomedical science have absolutely no inkling whatsoever about programming. Furthermore, Microsoft office is installed everywhere everywhere everywhere… like it leave it love it or hate it, it doesn’t matter… it’s true (at least for huge swaths of the demographic). So the first thing most people learn is excel and the last thing most…

Even newcomers? I've had my education in a engineering focused campus so that may skew it, but even not IT related area had programming courses to, at least, learn the basics. Do the newer researcher still show reluctance to move from Excel or is it an old guard kind of deal?

>why stick with Excel versus using python, R or what have you to treat the data? [...] Do the newer researcher still show reluctance to move from Excel

The feature of Excel that's underestimated/overlooked by skilled programmers of Python/R is that spreadsheets immediately expose an editable visual spatial canvas datagrid GUI.

I have decades of programming C/C++/C#/Python/etc and yet I create new spreadsheets and use them every day.

Spreadsheets are much faster than wiring up a datagrid in C# or C++ Qt gui widget editor or any other "true" programming language. Spreadsheets are also faster than using Python package Pandas to import a csv into a dataframe and view it in a Jupyter notebook. And last time I checked, displaying an output grid of cells in Jupyter is read-only and not 2-way editable like Excel.

And yes, the complaint is that "MS Excel calculations are not auditable, repeatable, version controlled, etc". All true, but it still doesn't change the fact that Python doesn't have an instant datagrid GUI. UI affordances also matter in viewing science data sets as well as financial budgets.

Re: Autocorrect errors in Excel still creating genomics headache

#37

Post by dspillett »

zozbot234 wrote: post #29
codetrotter wrote: post #28

That’s the price we pay. A reasonable alternative would be for example statically typing each column or row in the sheet. So in one sheet column A contains only floats, col B contains only text, col C contains only dates etc. And in another sheet col A may be date type, col B date type also, col C and D text, and col E monetary amounts. But this would come at the cost of not being able to mix types in column, and a l…

> A reasonable alternative would be for example statically typing each column or row in the sheet. Excel actually supports this out of the box. But it's one more option to set, and people get lazy.

Also those properties are lost if you transfer the data to another format then back, or don't properly survive copy/cut/paste operations (often cutting will remove everything about a cell including options such as these).

As well as being easy to be lazy and not use the options, it is easy to accidentally undo them also.

Re: Autocorrect errors in Excel still creating genomics headache

#38

Post by PedroBatista »

I generally am very critical of my own gang ( programmers ) for being so into their own bubble they completely lose perspective of the real World, and with that ignorance comes the usual arrogance.

But this time I feel we should give the business to the geneticists. The Excel "problem" is known and has been known for a long time, finance guys and policy makers were maybe the first big ones to "discover" this for slightly different reasons.

Then WHY on Earth these top professionals, elites, "creme de la creme" people continue on this path? And it's not like they love Excel, quite the contrary. Yes, the main reason is they are not programmers so they need to find a way to hack something to solve their problem. The problem with their "problem" solving is they are being slobs in a work that requires the EXACT opposite.

"Oh now we need to hire a programmer too?" - Yes, you do.

"But we don't have the budget for it" - Yes, you do. You see, money is never enough for anything, it's a question of priorities and when you blow millions of dollars with fancy experiments to ruin your career because of Excel, maybe that Python/R/Julia guy was a bargain after all.

I know some of these people, and under the aura of lab coats and distinguished professors it's just some guy/gal trying to crank that paper ASAP to keep the hamster wheel spinning AND they are being sloppy about the data and process AND most of them know it.

Sorry, but no excuses.

Re: Autocorrect errors in Excel still creating genomics headache

#39

Post by Closi »

planet-and-halo wrote: post #11

Serious question, is this a viable area for a startup product? Obviously nothing will easily displace Excel for general spreadsheet work, but I could see room for a domain-specific spreadsheet that has core features and supports just a basic set of functions relevant to genomics, + a "we won't fuck up your data" feature.

I think the size and scope of Excel is hard to replicate, however it sounds like it would make a great plugin.

Re: Autocorrect errors in Excel still creating genomics headache

#40

Post by masklinn »

planet-and-halo wrote: post #11

Serious question, is this a viable area for a startup product? Obviously nothing will easily displace Excel for general spreadsheet work, but I could see room for a domain-specific spreadsheet that has core features and supports just a basic set of functions relevant to genomics, + a "we won't fuck up your data" feature.

Probably not: people use excel because it's available, it's what they know, and it's what their peers use.

It's been going for more than half a decade now, and genomicists apparently would rather rename genes than stop using excel...

Post reply on HN