CSV is so deceptively simple that people don't care understanding it. I wasted countless hours working around services providing non-escaped data that off the shelf parsers could not parse.
A love letter to the CSV format
381–390 of 711 posts
Re: A love letter to the CSV format
#382CSV is ever so elegant but it has one fatal flaw - quoting has "non-local" effects, i.e. an extra or missing quote at byte 1 can change the meaning of a comma at byte 1000000. This has (at least) two annoying consequences: 1. It's tricky to parallelise processing of CSV. 2. A small amount of data corruption can have a big impact on the readability of a file (one missing or extra quote can bugger the whole thing up).…
I don't understand why CSV became a thing when TSV, or a format using the nowadays weird ASCII control characters like start/end of text, start of heading, horizontal/vertical tab, file/group/record/unit separator. It seems many possible designs would've avoided the quoting chaos and made parsing sort of trivial.
It's a lot easier to type comma than control characters and it's a lot easier to view comma than a tab (which might look like a space).
For automated serialization, plain text formats won out, because they are easy to implement a minimal working solution (both import and export) and more importantly, almost all systems agree on what plain text is.
We don't really have Apple formatted text, that will show up as binary on windows. Especially if you are just transferring id's and numbers, those will fall within ascii and that will work even if you are expecting unicode.
Re: A love letter to the CSV format
#383The post should at least mention in passing the major problem with CSV: it is a "no spec" family of de-facto formats, not a single thing (it is an example of "historically grown"). And omission of that meams I'm going to have to call this our for its bias (but then it is a love letter, and love makes blind...). Unlike XML or JSON, there isn't a document defining the grammar of well-formed or valid CSV files, and ther…
Waiting for someone to write a love letter to the infamous Windows INI file format
I actually miss that. It was nice when settings were stored right alongside your software, instead of being left behind all over a bloated registry. And the format was elegant, if crude.
I wrote my own library for encoding/writing/reading various datatypes and structure into ini's, in a couple different languages, and it served me well for years.
Re: A love letter to the CSV format
#384Earlier quoted context omitted.
Remove the [] characters and you've invented CSV with excel style quoting.
Almost, except the way Excel-style quoting works with newlines sucks - you end up with rows that span multiple lines, so you can't split on newline to get individual rows. With JSON those new lines are \n characters which are much easier to work with.
I ended up parsing the XML format instead of the CSV format when handling paste from Excel due to the newlines issue.
CSV seemed so simple but after numerous issues, a cell with both newline and " made me realize I should keep the little hair I had left and put in the work to parse the XML.
It's not great either, with all its weird tags, but at least it's possible to parse reliably.
Re: A love letter to the CSV format
#385The post should at least mention in passing the major problem with CSV: it is a "no spec" family of de-facto formats, not a single thing (it is an example of "historically grown"). And omission of that meams I'm going to have to call this our for its bias (but then it is a love letter, and love makes blind...). Unlike XML or JSON, there isn't a document defining the grammar of well-formed or valid CSV files, and ther…
Waiting for someone to write a love letter to the infamous Windows INI file format I actually miss that. It was nice when settings were stored right alongside your software, instead of being left behind all over a bloated registry. And the format was elegant, if crude. I wrote my own library for encoding/writing/reading various datatypes and structure into ini's, in a couple different languages, and it served me well…
I feel like YAML is a spiritual successor to the .ini, since it shares a notable ideal of simple human readability/writability.
Re: A love letter to the CSV format
#386CSV is ever so elegant but it has one fatal flaw - quoting has "non-local" effects, i.e. an extra or missing quote at byte 1 can change the meaning of a comma at byte 1000000. This has (at least) two annoying consequences: 1. It's tricky to parallelise processing of CSV. 2. A small amount of data corruption can have a big impact on the readability of a file (one missing or extra quote can bugger the whole thing up).…
JSON serialized without extra white space with one line per record is superior to CSV. If you want CSV-ish, enforce an array of strings for each record. Or go further with actual objects and non-string types. You can even jump to an arbitrary point and then seek till you see an actual new line as it’s always a record boundary. It’s not that CSV is an invalid format. It’s that libraries and tools to parse CSV tend to…
> JSON [...] with one line per record
Couple of standards that I know of that does this, primarily intended for logging:
Really easy to work with in my experience.
Sure some space is usually wasted on keys but compression takes care of that.
Re: A love letter to the CSV format
#387Earlier quoted context omitted.
Waiting for someone to write a love letter to the infamous Windows INI file format I actually miss that. It was nice when settings were stored right alongside your software, instead of being left behind all over a bloated registry. And the format was elegant, if crude. I wrote my own library for encoding/writing/reading various datatypes and structure into ini's, in a couple different languages, and it served me well…
I feel like YAML is a spiritual successor to the .ini, since it shares a notable ideal of simple human readability/writability.
Whenever I ask myself "should I use YAML?" I answer myself "Norway".
Re: A love letter to the CSV format
#388The post should at least mention in passing the major problem with CSV: it is a "no spec" family of de-facto formats, not a single thing (it is an example of "historically grown"). And omission of that meams I'm going to have to call this our for its bias (but then it is a love letter, and love makes blind...). Unlike XML or JSON, there isn't a document defining the grammar of well-formed or valid CSV files, and ther…
Waiting for someone to write a love letter to the infamous Windows INI file format I actually miss that. It was nice when settings were stored right alongside your software, instead of being left behind all over a bloated registry. And the format was elegant, if crude. I wrote my own library for encoding/writing/reading various datatypes and structure into ini's, in a couple different languages, and it served me well…
TOML is nice like that... elegant like INI, only with lists.
Re: A love letter to the CSV format
#389Earlier quoted context omitted.
Clearly they should have gone with BEL as the delimiter. printf "alice\007london\007uk\nbob\007paris\007france\n" > data.bsv I'm hoping no reasonable person would ever use BEL as punctuation or decimal separator.
On the off chance you're not being facetious, why not ASCII 0 as a delimiter? (This is a rhetorical question.)
ASCII has characters more or less designed for this
0x1C - File Separator
0x1D - Group Separator
0x1E - Record Separator
0x1F - Unit Separator
So I guess 1F would be the "comma" and 1E would be the "newline."
Re: A love letter to the CSV format
#390Earlier quoted context omitted.
Waiting for someone to write a love letter to the infamous Windows INI file format I actually miss that. It was nice when settings were stored right alongside your software, instead of being left behind all over a bloated registry. And the format was elegant, if crude. I wrote my own library for encoding/writing/reading various datatypes and structure into ini's, in a couple different languages, and it served me well…
I feel like YAML is a spiritual successor to the .ini, since it shares a notable ideal of simple human readability/writability.
Given how YAML does magic and sometimes accidental type conversions of potentially nested objects, I think TOML is the well-defined sucessor to .ini