Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to share your data effectively

caitlinrivers.com

Post reply on HN
Print view View original

11–20 of 127 posts

Re: How to share your data effectively

#11

Post by yebyen »

TsiCClawOfLight wrote: post #7
yebyen wrote: post #3

There is lots of bad advice in this article. OK maybe the article is shorter than I thought. Here, this is bad: Use short variable names with no whitespace. Underscores are usually a safe bet, so instead of "Number of new tuberculosis cases" use "incident_tb" While it might be clear to the domain experts, I had no idea that "incident_tb" meant "Number of new tuberculosis cases" other than that it was placed next to t…

This can be helpful advice for many people. Why bash him? It's good advice. I would understand "incident_tb" in health context, and I'm 17. So a readme should be plenty, right?

Yes, honestly I read down to that point and began writing my comment, assumed there would be lots more bad advice to follow, then the article went and ended on me. The README is the correct solution in this case.

Re: How to share your data effectively

#12

Post by yebyen »

mrtriangle wrote: post #5
yebyen wrote: post #3

There is lots of bad advice in this article. OK maybe the article is shorter than I thought. Here, this is bad: Use short variable names with no whitespace. Underscores are usually a safe bet, so instead of "Number of new tuberculosis cases" use "incident_tb" While it might be clear to the domain experts, I had no idea that "incident_tb" meant "Number of new tuberculosis cases" other than that it was placed next to t…

I agree with you, I actually had a client who demanded that the data in their project be done in excel (with multiple files ranging from 1-3 gb in size!). Both myself and my partner tried exhaustively to use a database or even JSON/XML solutions but the client kept repeating "in the real world business is done in excel!" Needless to say it was a nightmare.

That person is obviously bat-shit crazy. I am very surprised to hear that Excel or any other spreadsheet program will open multi-gigabyte files without exploding.

Re: How to share your data effectively

#13

Post by dav-id »

mrtriangle wrote: post #5
yebyen wrote: post #3

There is lots of bad advice in this article. OK maybe the article is shorter than I thought. Here, this is bad: Use short variable names with no whitespace. Underscores are usually a safe bet, so instead of "Number of new tuberculosis cases" use "incident_tb" While it might be clear to the domain experts, I had no idea that "incident_tb" meant "Number of new tuberculosis cases" other than that it was placed next to t…

I agree with you, I actually had a client who demanded that the data in their project be done in excel (with multiple files ranging from 1-3 gb in size!). Both myself and my partner tried exhaustively to use a database or even JSON/XML solutions but the client kept repeating "in the real world business is done in excel!" Needless to say it was a nightmare.

I don't have the full picture here but by moving away from an Excel format you are also potentially taking away a lot of control they have themselves over managing the data and working the data which Excel is extremely good at with little to no programming.

Re: How to share your data effectively

#14

Post by duncans »

mrtriangle wrote: post #5
yebyen wrote: post #3

There is lots of bad advice in this article. OK maybe the article is shorter than I thought. Here, this is bad: Use short variable names with no whitespace. Underscores are usually a safe bet, so instead of "Number of new tuberculosis cases" use "incident_tb" While it might be clear to the domain experts, I had no idea that "incident_tb" meant "Number of new tuberculosis cases" other than that it was placed next to t…

I agree with you, I actually had a client who demanded that the data in their project be done in excel (with multiple files ranging from 1-3 gb in size!). Both myself and my partner tried exhaustively to use a database or even JSON/XML solutions but the client kept repeating "in the real world business is done in excel!" Needless to say it was a nightmare.

Maybe your client was wanting ubiquitous Excel-compatible export/import - not for the back end to actually be Excel? Making any table in your app exportable to at least CSV is pretty straightforward.

Re: How to share your data effectively

#15

Post by ambiate »

mrtriangle wrote: post #5
yebyen wrote: post #3

There is lots of bad advice in this article. OK maybe the article is shorter than I thought. Here, this is bad: Use short variable names with no whitespace. Underscores are usually a safe bet, so instead of "Number of new tuberculosis cases" use "incident_tb" While it might be clear to the domain experts, I had no idea that "incident_tb" meant "Number of new tuberculosis cases" other than that it was placed next to t…

I agree with you, I actually had a client who demanded that the data in their project be done in excel (with multiple files ranging from 1-3 gb in size!). Both myself and my partner tried exhaustively to use a database or even JSON/XML solutions but the client kept repeating "in the real world business is done in excel!" Needless to say it was a nightmare.

This is seen all the time in Accounting departments. GAAP and STAT financials in spreadsheets with millions of linked documents and macros. A large amount of this deals with business logic being tied directly in Excel. Their data and the constraints around it are embedded into the spreadsheet. Segregating the data from the spreadsheet means adapting the business logic from Excel and maintaining it in two places. (Source: I work with an accounting department that regularly deals with spreadsheets that take anywhere from 30 minutes to 4 hours to load/process/filter).

Re: How to share your data effectively

#16

Post by jeffasinger »

mrtriangle wrote: post #5
yebyen wrote: post #3

There is lots of bad advice in this article. OK maybe the article is shorter than I thought. Here, this is bad: Use short variable names with no whitespace. Underscores are usually a safe bet, so instead of "Number of new tuberculosis cases" use "incident_tb" While it might be clear to the domain experts, I had no idea that "incident_tb" meant "Number of new tuberculosis cases" other than that it was placed next to t…

I agree with you, I actually had a client who demanded that the data in their project be done in excel (with multiple files ranging from 1-3 gb in size!). Both myself and my partner tried exhaustively to use a database or even JSON/XML solutions but the client kept repeating "in the real world business is done in excel!" Needless to say it was a nightmare.

This.

When moving from a mostly broken .xls data export system to a CSV file with the same data, we got lots of complaints from customers that it was too hard to use.

Re: How to share your data effectively

#19

Post by yebyen »

jeffasinger wrote: post #16
mrtriangle wrote: post #5

Earlier quoted context omitted.

I agree with you, I actually had a client who demanded that the data in their project be done in excel (with multiple files ranging from 1-3 gb in size!). Both myself and my partner tried exhaustively to use a database or even JSON/XML solutions but the client kept repeating "in the real world business is done in excel!" Needless to say it was a nightmare.

This. When moving from a mostly broken .xls data export system to a CSV file with the same data, we got lots of complaints from customers that it was too hard to use.

CSV files ARE broken in Excel. Have you ever tried importing a column of data with numbers that contain leading zeros / are not meant as numbers? There are at least 5 ways to do it wrong.

I know how to do this in LibreOffice (requires an extra step above and beyond double clicking) but honestly I do not know how to do it in Excel, and I would not attempt to learn or teach this knowledge to a person who does not have experience with a different spreadsheet tool first.

I will send them .xls formatted files instead.

Re: How to share your data effectively

#20

Post by onemorepassword »

My preference is .csv, because it can be read by almost any program.

Except the one most used for it: Excel. Excel sucks at csv, especially if the recipients have various internationalized versions. It can't deal properly with multi-line strings and spaces, but especially the internationalization is hell (semi-colons instead of commas, decimal separators, date formats).

Many times have I dealt with CSV-exports that could be read in any software except client's Excel versions.

Post reply on HN