Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autocorrect errors in Excel still creating genomics headache

nature.com

Post reply on HN
Print view View original

91–100 of 108 posts

Re: Autocorrect errors in Excel still creating genomics headache

#91

Post by cyanydeez »

tssva wrote: post #86

This isn't a Microsoft or Excel issue. It is a choosing a spreadsheet issue. LibreOffice and Google Sheets for instance by default also try to determine what type of data is being entered. The issue is choosing the incorrect tool or if you have no choice but to use a spreadsheet failing to learn how your tools work.

unfortunately, the reality is backwards compatibility with everyone who came before you, sets your budgets and expects to review data with minimal upstart.

people like spreadsheets because it provides very immediate feedback

Re: Autocorrect errors in Excel still creating genomics headache

#92

Post by cyanydeez »

bserge wrote: post #80
ironmagma wrote: post #55

Earlier quoted context omitted.

People don't choose to use Microsoft products, the software is chosen for them by their organization. For a problem like this, the existence of the article may very well be the first step to switching to another solution. The first step in that process is raising awareness that there is a problem.

If only the organization was run by people.

we stand on the shoulders of excel users, thats the reality.

Re: Autocorrect errors in Excel still creating genomics headache

#93

Post by OptionX »

jasode wrote: post #36
OptionX wrote: post #23

Earlier quoted context omitted.

Even newcomers? I've had my education in a engineering focused campus so that may skew it, but even not IT related area had programming courses to, at least, learn the basics. Do the newer researcher still show reluctance to move from Excel or is it an old guard kind of deal?

>why stick with Excel versus using python, R or what have you to treat the data? [...] Do the newer researcher still show reluctance to move from Excel The feature of Excel that's underestimated/overlooked by skilled programmers of Python/R is that spreadsheets immediately expose an editable visual spatial canvas datagrid GUI . I have decades of programming C/C++/C#/Python/etc and yet I create new spreadsheets and us…

I'm not saying Excel is without its benefits or a bad tool, just perhaps shouldn't be the only one.

And I was wondering if people like in the non-engineering but scientific fields, as the ones discussed in post, are exposed to the alternatives during their formation years.

If one chooses to use Excel because its the best tool great, but if its because its the perceived only option that's probably counterproductive.

Re: Autocorrect errors in Excel still creating genomics headache

#94

Post by KMnO4 »

UnFleshedOne wrote: post #53
tcskeptic wrote: post #19

We need a clippy type feature "It looks like you are working on genomics data, should I turn off all the data destroying auotcorrect features?"

"It looks like you are working on genomics data, should I direct you to a search engine to find appropriate tooling instead?"

Heh, sure, if you have $10k+ per user.

(Maybe genomics isn’t as bad as proteomics)

Re: Autocorrect errors in Excel still creating genomics headache

#95

Post by acomjean »

qayxc wrote: post #25
gerdesj wrote: post #5

Earlier quoted context omitted.

"That’s something Purdie says she doesn’t have time for. She has adapted to Excel’s quirks, adding apostrophes before commonly affected genes to prevent the conversion, or pre-formatting spreadsheet cells before importing data. “It’s one of those things that I just accept,” she says."

Wait, if she knows how to avoid the issue, how is it still an issue?

I work in bioinformatics and have seen the issue.

You get data from a lot of sources, and people (like me) just sometimes forget to check the spreadsheet. Often a csv file so it’s not clear it’s been generated by excel. It’s also only a problem for gene symbols (not entrez gene IDs or flybase gene numbers). Also the datasets are big, so there are 15 genes in error over the whole dataset it can be hard to spot.

Honestly genes are a constantly moving target and are kinda a pain to write software for. They get new names, split and merge over time…

Re: Autocorrect errors in Excel still creating genomics headache

#96

Post by ironmagma »

bserge wrote: post #80
ironmagma wrote: post #55

Earlier quoted context omitted.

People don't choose to use Microsoft products, the software is chosen for them by their organization. For a problem like this, the existence of the article may very well be the first step to switching to another solution. The first step in that process is raising awareness that there is a problem.

If only the organization was run by people.

To a first-order approximation, all people are not management.

Re: Autocorrect errors in Excel still creating genomics headache

#97

Post by setr »

breakfastduck wrote: post #77
ironmagma wrote: post #65

Earlier quoted context omitted.

People working in large organizations are frequently not even allowed to install software that isn’t pre-approved by N layers of management. The bottom line though is that Excel is made for data entry. If it’s not suitable for that use case, that’s a big problem with the product-market fit.

I cannot imagine a single piece of software that is more widespread and used without issue by huge swathes of people than excel. But one specific use case for genetics means its totally unsuitable & and big problem with the product. Got it.

?

Excel documents almost always have issues, independent of genomics[0]. I can probably just skim through my emails and find a bunch of problem spreadsheets all over.

Fundamentally, the problem is threefold.

1) excel tries to be smart, and this works in the easy case and breaks down in the edge cases. The edge cases also tend to appear with larger data entry as various one-offs, so it’s particularly insidious

2. Excel is difficult to validate (formulas are hidden, naming things properly is a bit of a hack, no real typechecking, data constraints can be set but it’s hard to check what’s set to what across all cells, etc)

3. The typical user / target market isn’t trained to use what facilities excel does happen to offer.

Ultimately, the problem is that excel doesn’t “scale”, and by default you get the wrong thing.

And even if you know how to set things up properly, it’s near impossible to be sure it stays proper.

[0] https://www.marketwatch.com/story/88-of-spreadsheets-have-er...

Re: Autocorrect errors in Excel still creating genomics headache

#98

Post by eitland »

jjk166 wrote: post #68
eitland wrote: post #64

Earlier quoted context omitted.

> Given the rarity that someone types "SEPT4" into excel and isn't typing a date, putting the option one level deep in a menu seems more appropriate than a top level button. Given the rarity that someone has pasted 100 000 rows into and want them to be autoborked disabling autoborking, at least for pasted data seems appropriate enough for me ;-)

I would expect dates to show up in large datasets pasted into excel orders of magnitude more often than genes. For the people who use excel, autoformatting is overwhelmingly the desired default behavior.

> For the people who use excel, autoformatting is overwhelmingly the desired default behavior.

Seriously? Also for pasted series? Also when the rest of the column doesn't match?

Re: Autocorrect errors in Excel still creating genomics headache

#99

Post by qayxc »

acomjean wrote: post #95
qayxc wrote: post #25

Earlier quoted context omitted.

Wait, if she knows how to avoid the issue, how is it still an issue?

I work in bioinformatics and have seen the issue. You get data from a lot of sources, and people (like me) just sometimes forget to check the spreadsheet. Often a csv file so it’s not clear it’s been generated by excel. It’s also only a problem for gene symbols (not entrez gene IDs or flybase gene numbers). Also the datasets are big, so there are 15 genes in error over the whole dataset it can be hard to spot. Honest…

What really interests me is why your field is still using untyped CSV data for exchanging information, instead of say XML.

And why hasn't there been a collaboration between professionals to fix this on an organisational level (like agreeing on a spreadsheet template or sane method for data exchange)?

Is it really every lab for themselves with MS Excel and CSV of all things being the lowest common denominator?

Re: Autocorrect errors in Excel still creating genomics headache

#100

Post by netizen-936824 »

quantified wrote: post #84
BugsJustFindMe wrote: post #78

Earlier quoted context omitted.

> Excel is used as a database/storage/interchange format, especially after the initial analysis by someone who uses python or R. Bioinformatician does the analysis Sometimes, but the situation is in reality worse than that. Excel is also used as the gold standard database/storage/interchange format of record for random shit that clinical researchers have typed in by hand whether directly or transcribed from other not…

For the finance and business office worker, it seems to have traction. Just like auto-creating an emoji when you type a : character. Excel is for offices, not specializations of scientists. Bummer.

So maybe we need better software for scientists? Sounds like a hole in the market
Post reply on HN