A love letter to the CSV format • Hacker News Board

Re: A love letter to the CSV format

#461

Post by InsideOutSanta » Thu, Mar 27, 2025, 9:34 AM UTC

This is particularly funny because I just received a ticket saying that the CSV import in our product doesn't work. I asked for the CSV, and it uses a semicolon as a delimiter. That's just what their Excel produced, apparently. I'm taking their word for it because... Excel.

To me, CSV is one of the best examples of why Postel's Law is scary. Being a liberal recipient means your work never ends because senders will always find fun new ideas for interpreting the format creatively and keeping you on your toes.

Re: A love letter to the CSV format

#462

Post by Timwi » Thu, Mar 27, 2025, 9:39 AM UTC

I am annoyed that comma won out as the separator. Tab would have been a massively better choice. Especially for those of us who have discovered and embraced elastic tabstops. Any slightly large CSV is unreadable and uneditable because you can't easily see where the commas are, but with tabs and elastic tabstops, the whole thing is displayed as a nice table.

(That is, of course, assuming the file doesn't contain newlines or other tabs inside of fields. The format should use \t \n etc for those. What a missed opportunity.)

Re: A love letter to the CSV format

#463

Post by IanCal » Thu, Mar 27, 2025, 9:47 AM UTC

boogheta wrote: post #454

IanCal wrote: post #445

Earlier quoted context omitted.

The byte for a capital I is the same as the start for an odd file format, slyk maybe? Excel has (or did if they finally fixed it) for years decided this was enough to assume the file (called .csv) cannot possibly be csv but must actually be slyk. It then parses it as such, and is shocked to find your slyk file is totally broken!

It sounds to me like as often the problem here is Excel, not CSV

Clients don't particularly make the distinction, and in a way nor should they - they can't open your file.

Re: A love letter to the CSV format

#464

Post by Fokamul » Thu, Mar 27, 2025, 9:50 AM UTC

Timwi wrote: post #462

I am annoyed that comma won out as the separator. Tab would have been a massively better choice. Especially for those of us who have discovered and embraced elastic tabstops. Any slightly large CSV is unreadable and uneditable because you can't easily see where the commas are, but with tabs and elastic tabstops, the whole thing is displayed as a nice table. (That is, of course, assuming the file doesn't contain newli…

CSV have multiple different separators. Eg. Excel defaults to different separators based on locale. Like CZ locale, it uses commas in numbers instead of dot, so CSV uses semicolon as default separator.

Re: A love letter to the CSV format

#465

Post by Ygg2 » Thu, Mar 27, 2025, 9:53 AM UTC

HelloNurse wrote: post #437

consp wrote: post #427

Earlier quoted context omitted.

YAML is readable? No way as there are too many ways to do the same thing and nested structures are unclear to the non trained eye (what is a list? What is nested?), let alone indentation in large files is an issue especially with the default 2 space unreadable standard so many people adhere to. YAML simple? It's sepc is larger than XML... Parsing of numbers and strings is ambiguous, leading zeros are not strings but…

Say "no" to YAML. As a string, if you can.

You can. YAML 1.2 is only 16 years old. Just old enough to drive. Norway problem has been solved for only 16 years.

Re: A love letter to the CSV format

#466

Post by sbergot » Thu, Mar 27, 2025, 9:53 AM UTC

thiht wrote: post #443

sramsay64 wrote: post #360

Earlier quoted context omitted.

In fairness there are also several ambiguities with JSON. How do you handle multiple copies of the same key? Does the order of keys have semantic meaning? jq supports several pseudo-JSON formats that are quite useful like record separator separated JSON, newline separated JSON. These are obviously out of spec, but useful enough that I've used them and sometimes piped them into a .json file for storage. Also, encoding…

> How do you handle multiple copies of the same key That’s unambiguously allowed by the JSON spec, because it’s just a grammar. The semantics are up to the implementation.

interestingly other people are answering the opposite in this thread.

Re: A love letter to the CSV format

#467

Post by Fokamul » Thu, Mar 27, 2025, 9:56 AM UTC

InsideOutSanta wrote: post #461

This is particularly funny because I just received a ticket saying that the CSV import in our product doesn't work. I asked for the CSV, and it uses a semicolon as a delimiter. That's just what their Excel produced, apparently. I'm taking their word for it because... Excel. To me, CSV is one of the best examples of why Postel's Law is scary. Being a liberal recipient means your work never ends because senders will al…

Of course, because there are locales which uses comma as decimal separator. So CSV in Excel then defaults to semicolon.

Another Microsoft BS, they should defaults to ENG locale in CSV, do a translation in background. And let user choose, if they want to save as different separator. Excel in every part of world should produce same CSV by default. Bunch of idiots.

Re: A love letter to the CSV format

#468

Post by boogheta » Thu, Mar 27, 2025, 9:57 AM UTC

afiori wrote: post #446

juliansimioni wrote: post #108

Earlier quoted context omitted.

What happens when you need to encode the newline character in your data? That makes splitting _either_ CSV or LDJSON files difficult.

Actually even whitespace-separated json would be a valid format and if you forbid json documents to be a single integer or float then even just concatenating json gives a valid format as JSON is a prefix free language. That is[0] if a string s is a valid JSON then there is no substring s[0..i] for i So you could just consume as many bytes you need to produce a json and then start a new one when that one is complete.…

« a single missing " can destroy the entire document » This is basically true for any data format, so really worse argument ever...

Re: A love letter to the CSV format

#469

Post by VMG » Thu, Mar 27, 2025, 10:02 AM UTC

dtech wrote: post #393

diekhans wrote: post #362

Earlier quoted context omitted.

Plus the 64-bit integer problem, really 52-bit integers, due to JS not having integers.

Most good parsers have an option to parse to integers or arbitrary precision decimals.

Agreed. Which means that Javascript does not have a good parser.

Re: A love letter to the CSV format

#470

Post by tobyhinloopen » Thu, Mar 27, 2025, 10:03 AM UTC

diekhans wrote: post #362

sramsay64 wrote: post #360

Earlier quoted context omitted.

In fairness there are also several ambiguities with JSON. How do you handle multiple copies of the same key? Does the order of keys have semantic meaning? jq supports several pseudo-JSON formats that are quite useful like record separator separated JSON, newline separated JSON. These are obviously out of spec, but useful enough that I've used them and sometimes piped them into a .json file for storage. Also, encoding…

Plus the 64-bit integer problem, really 52-bit integers, due to JS not having integers.

bigint exists