Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CSV 1.1 – CSV Evolved (for Humans)

csv11.github.io

Post reply on HN
Print view View original

21–30 of 81 posts

Re: CSV 1.1 – CSV Evolved (for Humans)

#21

Post by PakG1 »

Someone1234 wrote: post #17
Jaruzel wrote: post #2

I can't see this gaining any traction. There's nothing wrong with CSV as it is for basic data transfer. CSV had been unchanged now for so long, that any attempt to update it will fail - there are simply too many CSV parsers already implemented. If you are worried about people in the year 3000 understanding your data, add a .txt file alongside the CSV explaining the fields.

I agree. The biggest problem with CSV is Excel's terrible support. And while CSV is primarily used for data transfer, Excel could hypothetically be a great debug-viewer if it didn't do brain dead stuff like unrecoverably corrupt UPCs and other long numbers by default. There's an Excel Uservoice about this very issue from 2015: https://excel.uservoice.com/forums/304921-excel-for-windows-... They're "looking into it."…

I was surprised by how bad Microsoft's CSV support is in Excel. So much that I use Numbers or text editors for everything CSV, never do I use Excel. Excel is still great for a basic level of intensive data analysis though, so can't get away from it.

Re: CSV 1.1 – CSV Evolved (for Humans)

#23

Post by gwbas1c »

What problem is this trying to solve?

CSV's problems are the nature of a very flexible convention. It's so simple that everyone writes their own generators and parsers that are slightly different. That's what happens when you use a convention like csv.

Revving the spec won't help anything... Because csv is the kind of convention where no one reads the spec anyway!

Re: CSV 1.1 – CSV Evolved (for Humans)

#24

Post by paulryanrogers »

Someone1234 wrote: post #17
Jaruzel wrote: post #2

I can't see this gaining any traction. There's nothing wrong with CSV as it is for basic data transfer. CSV had been unchanged now for so long, that any attempt to update it will fail - there are simply too many CSV parsers already implemented. If you are worried about people in the year 3000 understanding your data, add a .txt file alongside the CSV explaining the fields.

I agree. The biggest problem with CSV is Excel's terrible support. And while CSV is primarily used for data transfer, Excel could hypothetically be a great debug-viewer if it didn't do brain dead stuff like unrecoverably corrupt UPCs and other long numbers by default. There's an Excel Uservoice about this very issue from 2015: https://excel.uservoice.com/forums/304921-excel-for-windows-... They're "looking into it."…

IIRC there is a hack where you put a single quote after the comma and Excel will treat it as a string. Unfortunately it doesn't help exporting to unknown systems which l likely don't do the same.

Re: CSV 1.1 – CSV Evolved (for Humans)

#26

Post by wlesieutre »

hodgesrm wrote: post #15
Lazare wrote: post #14

Earlier quoted context omitted.

> There's nothing wrong with CSV as it is for basic data transfer. I disagree. CSV is horribly underspecified and many parsers have conflicting ideas on how things should work. I've hit areas where, for example, an API was generating a CSV that another API could read, but NOT if it was opened and then re-saved by Excel first (just re-saving, not editing), because Excel was making some trivial change in the format tha…

To your point it's hard to see how the referenced article qualifies as a standard. It does not deal with escape characters or missing values, two places where CSV implementations tend to vary in random ways.

I've recently been dealing with some CSV data exported from Excel. One of the columns has lengths measured in inches or feet. A column that says

    14"
exports as

    , '"14"""',
It feels downright silly to me.

Re: CSV 1.1 – CSV Evolved (for Humans)

#27

Post by pishpash »

edejong wrote: post #7

Well, if you want to improve tabular data formats: 1. Add a version identifier / content-type on the first line! 2. Create a formal grammar for this CSV format 3. Specify preferred character-encoding 4. Provide some tooling (validation, CSV 1.1 => HTML, CSV => Excel) 5. Add the option to specify column type (string, int, date) 6. Specify ISO-8601 as the preferred date format 7. Allow 'reheading' the columns in the fi…

Or how about include the parser with the file? That's self specifying.

Re: CSV 1.1 – CSV Evolved (for Humans)

#28

Post by asaph »

legostormtroopr wrote: post #13

There are a ton of problems here: * From the documentation "No quotes needed for values ... Use dual quotes ... Use triple quotes" * You haven't solved the problem of describing what the columns are. In fact, its worse because you are encouraging people to put units in the field * Spaces matter! What if the data is literally " Word " vs "Word". This format makes them both the same. * For some reason, the header row i…

> Spaces matter! What if the data is literally " Word " vs "Word". This format makes them both the same.

If leading and/or trailing whitespace are significant, you can quote the value. Problem solved.

Having said that, almost always when you see data persisted with leading or trailing whitespace it should have been trimmed away before storage.

Re: CSV 1.1 – CSV Evolved (for Humans)

#29

Post by tomc1985 »

Someone1234 wrote: post #17
Jaruzel wrote: post #2

I can't see this gaining any traction. There's nothing wrong with CSV as it is for basic data transfer. CSV had been unchanged now for so long, that any attempt to update it will fail - there are simply too many CSV parsers already implemented. If you are worried about people in the year 3000 understanding your data, add a .txt file alongside the CSV explaining the fields.

I agree. The biggest problem with CSV is Excel's terrible support. And while CSV is primarily used for data transfer, Excel could hypothetically be a great debug-viewer if it didn't do brain dead stuff like unrecoverably corrupt UPCs and other long numbers by default. There's an Excel Uservoice about this very issue from 2015: https://excel.uservoice.com/forums/304921-excel-for-windows-... They're "looking into it."…

Fix it so now its easier for customers to not use Excel? Heretic!

Re: CSV 1.1 – CSV Evolved (for Humans)

#30

Post by _pwalsh »

ewheeler wrote: post #10

For those looking for a modern take on improving on the CSV format, I'd recommend Frictionless Data's Datapackage specification[1] which basically consists of a json file of metadata that accompanies a CSV file that describes column types, versions, sources, and how to validate correctness of the CSV's data. This allows for quite a lot of tooling and workflow improvements to CSV files without mucking with the CSV its…

One of the authors of the Frictionless Data specifications here. The spec directly relevant to CSV is Table Schema [1], and we’ve also got some nice tools that leverage Table Schema and the family of specifications, such as goodtables [2].

[1] https://frictionlessdata.io/specs/table-schema/ [2] https://github.com/frictionlessdata/goodtables-py

Post reply on HN