While it’s great to dream I hope the CSV file stays around until I retire. Still the best way to get ‘simple’ data out of excel and uploaded on the front end (papa parse). More complicated data most likely should be using another format. I guess there is always TSV if the CSV goes away :)
Time to retire the CSV?
131–140 of 594 posts
Re: Time to retire the CSV?
#132[deleted]
Re: Time to retire the CSV?
#133You will pry CSVs form my cold, dead hands.
When I have lots of data in multiple related tables where I would benefit from defined data types, I reach for HDF5 or SQLite, but there are so many nice things to say about a simple CSV for lots of simply structured data.
Super simple to stream compress/decompress, and the ability to use great CLI tools like xsv, and being able to peak at the data by simply calling less or head... it's just hard to beat.
Re: Time to retire the CSV?
#134CSV (well TSV) as a format is a simple as it gets: one special code as field separator, and another one as line/record terminator. Only that CSV/TSV use chars available in text editors rather than the proper (archaic) ASCII C1 codes. Whatever the author has against CSV, I feel like starting a war against CSV is crazy, since any alternative is going to be worse when the problem isn't the format as such, but folks misusing it or using it wrongly (eg. broken XML in RSS, broken JSON, whatever).
Re: Time to retire the CSV?
#135Earlier quoted context omitted.
As someone that works with maven and npm.... what? This is valid JSON [["bob", "jones", 1, 22], ["frank", "was", 32, 45]] That's unreadable and unparsable by a human? Not only is JSON often more parsable, because it's structured it also becomes a lot easier to query. I grep and awk xml and json stuff all the time. I also have the added bonus of being able to use `jq` for json content.
Thats just a CSV with extra steps.
Nope, that's CSV without the drawbacks of CSV. That's CSV that can have special characters and doesn't suffer from delimiter problems.
When someone says "Maybe we can fix CSV" this is what you should do instead of trying to "fix" CSV.
Re: Time to retire the CSV?
#136CSV was a thing long before I was born, so I'm not privy to how it came about. But at least in day-to-day work, the single biggest drawback of CSV in my experience is the fact that the comma and most of the other common delimiters occur regularly in real data, forcing all of the cumbersome escape sequences. To say nothing of someone misplacing a quote somewhere and throwing off the cell count. So, question to the gre…
> why didn't we use one of the dedicated Data Structure separator control codes (e.g. File/Group/Record/Unit separator) that were part of the ASCII standard?
Because you can't see them. CSV is, at its core, a text format. Using FS/GS/RS/US would effectively make it a binary format.
Also, what happens if one of those bytes appears in data?
The simple fact is you will never be able to come up with a text-based format that can handle all possible values without escape sequences, quoting, or length-encoding. And really, that's not that big a deal. It just means you have to write a simple state machine instead of using regex or your language's equivalent of split().
Re: Time to retire the CSV?
#137Earlier quoted context omitted.
As I mentioned down-thread, I can generate a CSV with a couple of fprintf statements and a loop. I definitely can't do that with .xlsx. There is almost zero friction to bolting CSV export capability to an existing system, which is part of why it's so popular.
> As I mentioned down-thread, I can generate a CSV with a couple of fprintf statements and a loop. And usually generate garbage for anything but the most trivial case, which really nobody gives a shit about. That's the main reason why CSV absolutely sucks too, you have to waste month diagnosing the broken shit you're given to implement the workarounds necessary to deal with it. > I definitely can't do that with .xlsx…
> That's the main reason why CSV absolutely sucks too [...]
Is it? I think you're absolutely right that naive points of view like the one you're responding to will lead to avoidable bugs, but I'm not so sure the problem is CSV so much as people who assume CSV is simple enough to parse or generate without using a library.
Re: Time to retire the CSV?
#138I don't agree with giving up csvs until the following conditions are met: 1) A truly open format is available and accessible. Csvs are textfiles. There is no system around that cannot open a textfile. If the format is binary or requires patents or whatever, then it's a non-starter. 2) Applications have a speed increase from using csvs. To wit, I loved csvs because often they finish preparing much faster than a "forma…
You're comparing CSVs to other spreadsheet document formats. But a CSV is not a spreadsheet. A CSV is raw data. (It's data that is restricted to a shape that enables it to be easily imported into a spreadsheet—but data nevertheless.) As such, it should be compared to other data formats—e.g. YAML, JSON Lines, etc. These other data formats all win on your #2 against CSV, as CSV is actually horrible at parse-time vs. ot…
> There's also Avro, which fails your point #1 (it's a binary format) but that binary format is a lossless alternate encoding of what's canonically a JSON document, and there are both simple CLI tools / and small, free, high-quality libraries that can map back and forth between the "raw" JSON document and the Avro-encoded file. At any time, you can decode the Avro-encoded file to text, to examine/modify it in a text editor.
Avro is not a lossless alternate encoding of what's canonically a JSON document. Yes Avro supports a JSON encoding, but it's not canonical.
In general though, you're point about Avro being able to be represented as text is valid, and applies to practically any binary formThere's also Avro, which fails your point #1 (it's a binary format) but that binary format is a lossless alternate encoding of what's canonically a JSON document, and there are both simple CLI tools / and small, free, high-quality libraries that can map back and forth between the "raw" JSON document and the Avro-encoded file. At any time, you can decode the Avro-encoded file to text, to examine/modify it in a text editor.at, which is why the whole "but it needs to be a text format" argument is garbage.
> The data-warehouse ecosystem already standardized on Avro as its data interchange format. And spreadsheets are just tiny data warehouses. So why not? ;)
I wish that the data-warehouse ecosystem standardized on anything. ;-)
That said, there are plenty of good reasons why a data-warehouse standard would not be advisable for spreadsheets.
Re: Time to retire the CSV?
#139I don't agree with giving up csvs until the following conditions are met: 1) A truly open format is available and accessible. Csvs are textfiles. There is no system around that cannot open a textfile. If the format is binary or requires patents or whatever, then it's a non-starter. 2) Applications have a speed increase from using csvs. To wit, I loved csvs because often they finish preparing much faster than a "forma…
OP and you gave me an idea : "The only true successor of CSV should be forward/backward compatible with any existing CSV variant" If we manage to write a spec that meet this criteria we'll have a powerful standard with easy adoption.
> If we manage to write a spec that meet this criteria we'll have a powerful standard with easy adoption.
So, a binary format consisting of: (1) a text data segment (2) and end of file character (3) a second text data segment with structured metadata describing the layout of the first text data segment, which can be as simple (in terms of meaning; the structure should be more constrained for machine readability) as “It’s some kind of CSV, yo!” to a description of specific CSV variations (headers? column data types? escaping mechanisms? etc.) or even specify that the main body is JSON, YAML, XML, etc. (which would probably often be detectable by inspection, but this removes any ambiguity).
Re: Time to retire the CSV?
#140Earlier quoted context omitted.
JSON and XML both make it easy to see what is going wrong and don't have near the same amount of drawbacks that CSV has.
Both are also not a good fit for columnar data at all.
I disagree. Json, in particular, can be nearly as compact as CSV by storing the data as an array of arrays.
[[1,2,3],
[4,5,6]]