Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CSVs Are Kinda Bad. DSVs Are Kinda Good

matthodges.com

Post reply on HN
Print view View original

81–90 of 133 posts

Re: CSVs Are Kinda Bad. DSVs Are Kinda Good

#81

Post by disgruntledphd2 »

Kon-Peki wrote: post #53
throw0101d wrote: post #24

Earlier quoted context omitted.

[flagged]

> Excel If you’re deep in the Excel world, chances are extremely high that you also have access to SSMS, which has a really, really good data import tool that makes short work of nasty CSV files. The output of this tool doesn’t even have to be SQL Server, it will use any ODBC driver you’ve got installed; you can send the data to Excel or even a new, properly formatted CSV. And if you want a repeatable package, there…

> If you’re deep in the Excel world, chances are extremely high that you also have access to SSMS,

Can you provide more information about this?

Like, I'd reinstall windows if this would actually work on messy excel/CSV data.

Re: CSVs Are Kinda Bad. DSVs Are Kinda Good

#82

Post by joseda-hg »

OscarCunningham wrote: post #50
teddyh wrote: post #6

The article talks about reading and parsing CSV data of unknown variants, but then skips to the solution being using a different format altogether. But you can only switch to a different format if you are producing data, not if you are reading it! And if you are in control of producing data, just produce strict RFC 4180-compliant CSV data and everybody will be able to read it just fine. There is no need to make your…

I just had a look at RFC 4180. This is the grammar they suggest: > file = [header CRLF] record *(CRLF record) [CRLF] I find it kind of wild that you have to have at least one record. Suppose I have a program that lists the events that occurred on a given day. How do I represent the fact that the program ran successfully but that there weren't any events on that day?

Event - Loging Started Aditionally, Loging Ended (for the day) allows you to check for intermediate crashes and startup errors, kinda like a pulse / health check

Not necessarily how I'd do it intuitively, but doesn't seem that crazy

Re: CSVs Are Kinda Bad. DSVs Are Kinda Good

#83

Post by disgruntledphd2 »

ElevenLathe wrote: post #73
memtet wrote: post #72

Earlier quoted context omitted.

Right, the author skipped right over human-readable TSV files which play nicely with sed/awk/grep/sort pipelines, and are supported by all CSV parsers and spreadsheet software.

TSV is also my go-to when mucking around on the command line. Perfect for noodling with data before you have to put together an Excel file to show to management.

The problem is that people (non-technical mostly), put tabs in fields, and then you have all the problems that the article notes.

Re: CSVs Are Kinda Bad. DSVs Are Kinda Good

#84

Post by ElevenLathe »

disgruntledphd2 wrote: post #83
ElevenLathe wrote: post #73

Earlier quoted context omitted.

TSV is also my go-to when mucking around on the command line. Perfect for noodling with data before you have to put together an Excel file to show to management.

The problem is that people (non-technical mostly), put tabs in fields, and then you have all the problems that the article notes.

I personally find that this happens (a lot) less often than with commas or quote characters.

Re: CSVs Are Kinda Bad. DSVs Are Kinda Good

#85

Post by jltsiren »

zelphirkalt wrote: post #16
dflock wrote: post #11

If only... Every time I have to do any major work with CSVs, I re-lament this exact thing. I think the only way this could ever become more widespread is to fix all the open source tooling so that it's eventually just supported everywhere - then keep evangelizing for... ~30 yrs. Probably you should also register a new mime type and extension and make it a new thing - don't overload .CSV any further - but make the sam…

If time could be turned back, a good idea would be to make CSV mean CSV. Not semicolon separated values, not any other thing separated values, but only comma separated values. To not overload the name in the first place.

And I would rename the format to SSV and make semicolon the separator. Comma is a terrible choice, because it's used as the decimal separator in many countries around the world.

Re: CSVs Are Kinda Bad. DSVs Are Kinda Good

#87

Post by disgruntledphd2 »

ElevenLathe wrote: post #84
disgruntledphd2 wrote: post #83

Earlier quoted context omitted.

The problem is that people (non-technical mostly), put tabs in fields, and then you have all the problems that the article notes.

I personally find that this happens (a lot) less often than with commas or quote characters.

That's fair, but it only takes one to mess up the rest of the file.

Re: CSVs Are Kinda Bad. DSVs Are Kinda Good

#88

Post by 7bit »

OscarCunningham wrote: post #50
teddyh wrote: post #6

The article talks about reading and parsing CSV data of unknown variants, but then skips to the solution being using a different format altogether. But you can only switch to a different format if you are producing data, not if you are reading it! And if you are in control of producing data, just produce strict RFC 4180-compliant CSV data and everybody will be able to read it just fine. There is no need to make your…

I just had a look at RFC 4180. This is the grammar they suggest: > file = [header CRLF] record *(CRLF record) [CRLF] I find it kind of wild that you have to have at least one record. Suppose I have a program that lists the events that occurred on a given day. How do I represent the fact that the program ran successfully but that there weren't any events on that day?

That isn't a problem of the file format. That's a problem of your process.

Re: CSVs Are Kinda Bad. DSVs Are Kinda Good

#90

Post by NikkiA »

throw0101d wrote: post #24
teddyh wrote: post #6

The article talks about reading and parsing CSV data of unknown variants, but then skips to the solution being using a different format altogether. But you can only switch to a different format if you are producing data, not if you are reading it! And if you are in control of producing data, just produce strict RFC 4180-compliant CSV data and everybody will be able to read it just fine. There is no need to make your…

[flagged]

Most likely the easiest solution is to use a python library to read the excel file, and another to export the RFC-compliant data.
Post reply on HN