I would love to drop csv for something else but I won't trade human readable for a binary format.
In fact, I will go even further and say I want something mergeable by source control.
Time to retire the CSV?
141–150 of 594 posts
Re: Time to retire the CSV?
#142I don't agree with giving up csvs until the following conditions are met: 1) A truly open format is available and accessible. Csvs are textfiles. There is no system around that cannot open a textfile. If the format is binary or requires patents or whatever, then it's a non-starter. 2) Applications have a speed increase from using csvs. To wit, I loved csvs because often they finish preparing much faster than a "forma…
To point 1, I'd argue that a SQLite database is a great next step beyond CSV despite being a binary format.
No, SQLite's dynamic data types would silently coerce data just like opening a CSV directly with Excel does.
The advantage of CSV is that it's as accurate as your plain text representation of your data can be. Since binary data can be represented by character data, that's 100% accurate. As soon as you introduce a storage format that has made assumptions about the type of data being stored, you've lost flexibility.
SQLite is not intended for data serialization. It's intended for data storage to be read back by essentially the same application in the same environment.
Re: Time to retire the CSV?
#143As the author of a CSV munging tool (CSVfix) I think most of the problems with CSV could be fixed if people producing CSV output, and people reading CSV input obeyed the rules of the RFC. Sadly, most people don't, and any textual output or input is routinely described as CSV, when it is nothing of the sort - even to the extent of not being comma-separated!
Is there something like CSVlint? Would be useful particular in the output stage, a kind of functional runtime test to make sure your program's output is valid and consumable.
Validating CSV syntax is kind of hard - in the worst case you would have to read the entire CSV file.
Re: Time to retire the CSV?
#144> While many programs can’t read or write Excel spreadsheets, almost anything can read and write CSVs, and a human can open a CSV file in any text editor and understand roughly what it contains.
Libra Office works in Windows.
Google Sheets does a great job too, no?
Re: Time to retire the CSV?
#145I never thought of Sqlite as a data transfer format before. I know it is frequenly used as application storage format. But to use as data transfer format is a really interesting use case. How much is it used like that in practise?
Re: Time to retire the CSV?
#146No.
Re: Time to retire the CSV?
#147I have always wondered why there is so little use of the separators in ASCII (the file, group, record and unit separators with codes 28-31). They seem perfect for the job and it would be easy to forbid inclusion of those characters in fields.
Re: Time to retire the CSV?
#148CSV (well TSV) as a format is a simple as it gets: one special code as field separator, and another one as line/record terminator. Only that CSV/TSV use chars available in text editors rather than the proper (archaic) ASCII C1 codes. Whatever the author has against CSV, I feel like starting a war against CSV is crazy, since any alternative is going to be worse when the problem isn't the format as such, but folks misu…
And then a field needs to contain one of those characters and everything breaks. And they're designed to be human editable, so the solution is to complicate parsing by using quoted fields instead of escape characters. Although someone might choose to use escape characters because there's no spec so they can do whatever they want.
Re: Time to retire the CSV?
#149CSV (well TSV) as a format is a simple as it gets: one special code as field separator, and another one as line/record terminator. Only that CSV/TSV use chars available in text editors rather than the proper (archaic) ASCII C1 codes. Whatever the author has against CSV, I feel like starting a war against CSV is crazy, since any alternative is going to be worse when the problem isn't the format as such, but folks misu…
I think if only editors and shells supported control codes better and gave them a sane visual representation, we could see a revival for them. Feels like a waste to have them and not to be used.
Re: Time to retire the CSV?
#150Earlier quoted context omitted.
OP and you gave me an idea : "The only true successor of CSV should be forward/backward compatible with any existing CSV variant" If we manage to write a spec that meet this criteria we'll have a powerful standard with easy adoption.
> If we manage to write a spec that meet this criteria we'll have a powerful standard with easy adoption. So, a binary format consisting of: (1) a text data segment (2) and end of file character (3) a second text data segment with structured metadata describing the layout of the first text data segment, which can be as simple (in terms of meaning; the structure should be more constrained for machine readability) as “…
You got my vibe
Almost any current CSV parser, even the bad ones, tolerate a header line.
So it should be possible to define a compact and standardized syntax that is appended before the real header of the first cell (separator,encoding,decimal separator (often disregarded by most parsers but crucial outside USA),quote character,escape character,etc...). Following headers would just use special notation to inform on (data-type,length,comment).
Newest parsers would use theses clues, older ones would just append some manageable junk to headers.