Time to retire the CSV?

> "That’s what CSV is."

That's really not a serious argument against CSV. Since you paraphrase in a silly way, I can do it too! Your "argument" is "Badly formatted files exist, therefore CSV bad".

Everyone "against CSV" seems to be arguing against badly formatted CSV, and leaping to the conclusion that "CSV is just bad" without much more to say about it. I'm sorry that badly formatted CSV gave you a bad time, but the format is fine and gets its job done.

"It doesn't have x, y or z feature therefore no one should be using it ever" is kind of a dumb argument, honestly.

Actually, there is already a standard way for someone to type these with the control key. To be specific, they are:

  FS: ^\ GS: ^] RS: ^^ US: ^_

Since these delimiters are not much used, on many terminals ^\ is mapped to send SIGQUIT. Use 'stty quit undef' and you can then type this at the keyboard. This works on Linux and OS X, and likely most other systems. Note that with most keyboards you are also using shift for RS and US. For emacs, prefix with C-q so that it inserts the literal.

Other posters have noted that they have ASCII separators recently, I have as recently as 2018 internally at a FAANG. So they are not completely unknown in the wild.

Excel has its advantages, but it is funny the tools people choose to work with. The number of times Access would make more sense...

> XML, JSON, and YAML all have this issue, too.

Yes. Though to their credit, some of those work with her numbers, which at least gets you 4 bits out of every 8 bits.

> And I know what you're about to argue, but JSON's datetime format is not in the spec. The common JSON datetime format is convention, not standard.

I'm not sure what argument you thought I was making, or why that comment is relevant.

> All you've shown is that CSV has the same limitations that XML, YAML, and JSON have, and those three formats specifically designed and intended for data serialization. Yes, the other formats do have other advantages, but they don't eliminate those three limitations, either.

I'm not sure what your mean by "eliminate", or why you think it matters that there are other formats with the same design trade offs.

> This is for data serialization, which means it's going to potentially be used with data systems that are wholly foreign separating great distances or great timespans. What data serialization format are you comparing CSV to? What do you think CSV is actually used for?

CSV is used for a variety of purposes. The context of the article is using it for data transfer.

The claim was that it was a compact format for data transfer, which is demonstrably not true.

> Are you arguing for straight binary? You know that CSV, XML, YAML and JSON all grew out of the reaction to how inscrutable both binary files and fixed width files were in the 80s and 90s, right?

I'm not sure what "straight binary" means to you. JSON is, for the most part, a binary encoding standard (just not a particularly good one).

You've got the heritage a bit wrong, as XML was not originally designed for data transfer at all. It was an attempt to simplify the SGML document markup language, and the data transfer aspects were subsequently grafted on. JSON & YAML have a slightly more complicated heritage, but neither was intended as a data transfer format. They've all been pressed in to service for that purpose, for a variety of reasons, that can charitably described as tactically advantageous but strategically flawed.

> Binary has all sorts of lovely problems you get to work with like endianness and some systems getting confused if they encounter a mid-file EOF.

I don't know how to break this to you, but text formats can have endianess (in fact, insanely UTF-8 does!), and systems being confused and whether they are at EOF as well.

> Yes, you do end up with a wasted space, but the file is in plain text and ZIP compression is a thing if that's actually a concern.

Wouldn't ZIP be a binary format, with all the problems and concerns you have with binary formats?

So to summarize what you are saying... "CSV is a compact format because you can compress it if you are concerned and all the space it wastes".

Would it be fair to say then that any binary format is a text format because you can convert the binary into a text representation of the data? ;-)

I don't agree with the author's thesis, but you're wrong.

1. It is a very common use-case to load CSV data generated by an opaque system which did not specify the format exactly to you.

2. CSV files are _difficult_ to process I: Fields are of variable width, which is not specified elsewhere. So you can't start processing different records in parallel, since you have to process previous lines to figure out where they end. You can't even go by some clear line termination pattern, since the quoting and escaping rules, and the inconsistencies the author mentions, make it so that you may not be able to determine with certainty that what you're reading is an actual end-of-record or end-of-field. Maybe it's all just a dream^H^H^H^H^H quoted string?

3. CSV files are _difficult_ to process II: For the same reasons as the above, it is non-trivial and time consuming to recover from CSV generation mistakes, corruptions during transmission, or missing parts of the data.

However - if you get some guarantees and abot field width and about the non-use of field and record separators in quoted strings, then your life becomes much easier.