As the author of a CSV munging tool (CSVfix) I think most of the problems with CSV could be fixed if people producing CSV output, and people reading CSV input obeyed the rules of the RFC. Sadly, most people don't, and any textual output or input is routinely described as CSV, when it is nothing of the sort - even to the extent of not being comma-separated!
Time to retire the CSV?
61–70 of 594 posts
Re: Time to retire the CSV?
#62"In favour of what?", that is the matter. CSV is a format more for humans and less for machines, but that is the use case: a format that is good enough to be compiled by humans and read by machines. At the moment there aren't many alternatives.
I think this issue can be bypassed if a better format such as Arrow or Parquet can be used in Excel.
CSV isn't supported in Excel because Microsoft prefers it. MS prefers its own Excel format. Excel supports CSV because a large enough majority of Excel users want it, which means MS includes support for it. For MS to remove CSV support in favor of Arrow or Parquet support would not benefit MS at all, and would make many of its customers push back, so I don't see it happening.
Re: Time to retire the CSV?
#63"In favour of what?", that is the matter. CSV is a format more for humans and less for machines, but that is the use case: a format that is good enough to be compiled by humans and read by machines. At the moment there aren't many alternatives.
Objectively, CSV is terrible for humans despite being a plaintext format. No one reads CSVs: they're incomprehensible since the columns are not aligned with the headings. (You might be drawing an analogy with JSON, which is often human readable because it puts the keys right there next to the values). The best that can be said for its simplicity is that it's easy to write code that can dump data out in CSV format (an…
Surprised you would think that JSON is more human-readable, as the layout of that makes it visually quite appalling - not so bad when through a pretty-printer utility, admittedly, but may as well go for XML as another use case
Re: Time to retire the CSV?
#64More correctly titled, "I Don't Like CSV".
There are probably valid uses for CSV, but more often than not, it's the wrong choice.
As soon as data has any form of structure to it (and most data does). CSV complicates everything. Even for unstructured data, the problem of escape characters often shows it's ugly head. The moment your data contains a comma, tab, or space, you run into a nasty mess that, in the best case makes your system fail, and in the worst case silently adds corrupt data into the system.
Neither JSON nor XML suffer from that problem and both can easily be used in any scenario you'd use CSV. The only argument against either format is they are a bit more bulky than CSV.
Re: Time to retire the CSV?
#65Earlier quoted context omitted.
As I mentioned down-thread, I can generate a CSV with a couple of fprintf statements and a loop. I definitely can't do that with .xlsx. There is almost zero friction to bolting CSV export capability to an existing system, which is part of why it's so popular.
> As I mentioned down-thread, I can generate a CSV with a couple of fprintf statements and a loop. And usually generate garbage for anything but the most trivial case, which really nobody gives a shit about. That's the main reason why CSV absolutely sucks too, you have to waste month diagnosing the broken shit you're given to implement the workarounds necessary to deal with it. > I definitely can't do that with .xlsx…
> An xlsx file is just a bunch of XML files in a zip.
A bunch of XML files with opaque formats that MS constantly makes changes to to make its competitors have to keep chasing the format.
Re: Time to retire the CSV?
#66CSV was a thing long before I was born, so I'm not privy to how it came about. But at least in day-to-day work, the single biggest drawback of CSV in my experience is the fact that the comma and most of the other common delimiters occur regularly in real data, forcing all of the cumbersome escape sequences. To say nothing of someone misplacing a quote somewhere and throwing off the cell count. So, question to the gre…
Csv is only one form of delimited files. Tsv or Tab Separated Values is yet another. If you have either field delimiters or row delimiters (new line) in your data, change your delimiters and process appropriately.
Re: Time to retire the CSV?
#67I don't agree with giving up csvs until the following conditions are met: 1) A truly open format is available and accessible. Csvs are textfiles. There is no system around that cannot open a textfile. If the format is binary or requires patents or whatever, then it's a non-starter. 2) Applications have a speed increase from using csvs. To wit, I loved csvs because often they finish preparing much faster than a "forma…
OP and you gave me an idea : "The only true successor of CSV should be forward/backward compatible with any existing CSV variant" If we manage to write a spec that meet this criteria we'll have a powerful standard with easy adoption.
Given the infinity of CSV variants existing is the very reason why CSV blows goats, creating a spec compatible with that is
1. a fool's errand, CSV "variants" are not compatible with one another and regularly contradict one another (one needs not look any further than Excel's localised CSVs)
2. resulting in getting CSV anyway, which is a lot of efforts to do nothing
Re: Time to retire the CSV?
#68The author seems to be advocating for something like ISO 20022.
Which is great if you actually need it. But also complete overkill if you aren’t doing the kind of things it’s made for (and even then, still a bit painful)
CSV is at the other extreme but this is also why it’s so useful.
Re: Time to retire the CSV?
#69Earlier quoted context omitted.
CSV does not open in Excel, unless you play the usual tricks first...
Not sure what your basing that claim on. It is a natively supported format, even amidst its inconsistencies, and has been for decades. Are you referring to a particular format variant?
Excel definitely opens CSV files, and Windows file associations tend to set that up by default.
However, Excel then goes ahead with changing the content to often wrongly-assumed formats. My favourite hate is how it will convert mobile numbers from a string of digits to an unusable exponential-format number. Hey, it doesn't even set column widths to make the content visible!
Re: Time to retire the CSV?
#70Earlier quoted context omitted.
> As I mentioned down-thread, I can generate a CSV with a couple of fprintf statements and a loop. And usually generate garbage for anything but the most trivial case, which really nobody gives a shit about. That's the main reason why CSV absolutely sucks too, you have to waste month diagnosing the broken shit you're given to implement the workarounds necessary to deal with it. > I definitely can't do that with .xlsx…
> An xlsx file is just a bunch of XML files in a zip. A bunch of XML files with opaque formats that MS constantly makes changes to to make its competitors have to keep chasing the format.
Even if it were true it wouldn't matter a whit to the production side of the format, which is what "produce CSVs using fprintf" is: excel can consume them all.