21–30 of 108 posts
Can anyone with first hand experience on this tell shed some light on why stick with Excel versus using python, R or what have you to treat the data?
Try to tell someone who just needs to get a task done that he needs to spend the next 3 months learning programming from scratch vs using excel.
Earlier quoted context omitted.
Switch to what? Excel is everywhere. People are familiar with it. The same people are often not familiar with, and have never used, and may even be confused by “notepad.exe”.
These aren't someone's 85 year old grandparent getting a computer for the first time to look at pictures of their great grandchildren, these are people who have used a computer every day since they were 14 and have advanced STEM degrees. I think they are probably re-trainable.
Can anyone with first hand experience on this tell shed some light on why stick with Excel versus using python, R or what have you to treat the data?
Because an absolutely enormous number of people involved in all ends of genomics and biomedical science have absolutely no inkling whatsoever about programming. Furthermore, Microsoft office is installed everywhere everywhere everywhere… like it leave it love it or hate it, it doesn’t matter… it’s true (at least for huge swaths of the demographic). So the first thing most people learn is excel and the last thing most…
I've had my education in a engineering focused campus so that may skew it, but even not IT related area had programming courses to, at least, learn the basics.
Do the newer researcher still show reluctance to move from Excel or is it an old guard kind of deal?
Earlier quoted context omitted.
Because an absolutely enormous number of people involved in all ends of genomics and biomedical science have absolutely no inkling whatsoever about programming. Furthermore, Microsoft office is installed everywhere everywhere everywhere… like it leave it love it or hate it, it doesn’t matter… it’s true (at least for huge swaths of the demographic). So the first thing most people learn is excel and the last thing most…
Yeah. At some point though, people who want to do something that requires python or R, but refuse to learn it, and then use Excel (and make mistakes like the article mentions) should reconsider their choices. This is really out of character for me, but in this instance, relying on excel to do everything is a mistake by the user, not Microsoft's ridiculous data destroying import function. Use the right tool.
Step one should be learn your tool, no matter the tool.
There's several ways to avoid the issue in Excel. If someone working with Excel isn't able to learn that (heck, a simple template would suffice), I have no hope the same demographic would have any success with R or Python.
Can anyone with first hand experience on this tell shed some light on why stick with Excel versus using python, R or what have you to treat the data?
"That’s something Purdie says she doesn’t have time for. She has adapted to Excel’s quirks, adding apostrophes before commonly affected genes to prevent the conversion, or pre-formatting spreadsheet cells before importing data. “It’s one of those things that I just accept,” she says."
(You could argue that supplementary data should be in an open format, so you could automate this checking. I'm not against this, either.)
Can anyone with first hand experience on this tell shed some light on why stick with Excel versus using python, R or what have you to treat the data?
(1) Graduate students (who are the bulk of the scientific labor force) usually have some training in python, R, or Matlab, but are seldom practiced in applying it to real-world work. So if a lab standardizes on python, R, etc. the senior people have to do a lot of extra work to support the programming, rather than doing work with intrinsic scientific value. It's easier to teach the Excel quirks than to teach effective programming practices. Excel quirks are concrete, and "good practices" in programming are vague and situational.
(2) Python & R have their own pitfalls. It's easy to apply the wrong filters to a data set and compare subsets that aren't what you think you're comparing. Although when they go wrong, they go very wrong, and this is easier for a supervisor to notice.
(3) Excel has rich text formatting, graph embedding, and data types, so it's very useful as a human-readable data interchange & summary format even if you never do computation in it.
IMO neither programming languages nor Excel are a great fit for data analysis. Something purpose-made, like JMP or even GraphPad, is probably the better choice in most situations. Programming gets you automation but at the cost of high complexity. Since you still need to look at and think about the data there's a limit to how much time automation can save (Amdahl's law).
A reasonable alternative would be for example statically typing each column or row in the sheet.
So in one sheet column A contains only floats, col B contains only text, col C contains only dates etc. And in another sheet col A may be date type, col B date type also, col C and D text, and col E monetary amounts.
But this would come at the cost of not being able to mix types in column, and a lot of people want to be able to mix.
And that’s why we’re stuck
That’s the price we pay. A reasonable alternative would be for example statically typing each column or row in the sheet. So in one sheet column A contains only floats, col B contains only text, col C contains only dates etc. And in another sheet col A may be date type, col B date type also, col C and D text, and col E monetary amounts. But this would come at the cost of not being able to mix types in column, and a l…
Excel actually supports this out of the box. But it's one more option to set, and people get lazy.
That’s the price we pay. A reasonable alternative would be for example statically typing each column or row in the sheet. So in one sheet column A contains only floats, col B contains only text, col C contains only dates etc. And in another sheet col A may be date type, col B date type also, col C and D text, and col E monetary amounts. But this would come at the cost of not being able to mix types in column, and a l…
automatically "correcting" something that was correct based on a dumb hunch
It is really simple to remove the most embarrassing autocomplete errors on phones:
- just disable autocorrect,
- or preferably use another keyboard that just proposes fixes but leaves it to you to select the suggestion (I use SwiftKey and it does this, others probably exist too).
The same is not possible in Excel as far as I am aware:
You cannot turn of autoformatting.
The whole thing could be easily (on a ux level at least ;-) solved by introducing a single checkbox: "stop embarrassing me" with help text that says "turn off autoformatting".
Checking that box would leave values like you typed or pasted them in.
No need to select on a column basis.
Edit: It is also almost at the level of modern search engines that replace my very specific queries with generic queries about something unrelated or vaguely related.
Edit 2: it is already more than a year ago I guess since iPhones were caught changing the names of correctly spelled medications to the name of another unrelated medication. Sooner or later this incessant drive to be smarter than the user is going to cost lives I'm afraid.
When it comes to search it is already costing me many minutes lost a day, and that is just first order effects, before we get into all the things that doesn't get done because we give up on finding it.