11–20 of 81 posts
This is not VCS friendly. Resize any column and you have to rewrite 99% of the file, tracking changes becomes hell on earth.
* From the documentation "No quotes needed for values ... Use dual quotes ... Use triple quotes"
* You haven't solved the problem of describing what the columns are. In fact, its worse because you are encouraging people to put units in the field
* Spaces matter! What if the data is literally " Word " vs "Word". This format makes them both the same.
* For some reason, the header row is removed. The only thing in a CSV that provides structure is gone. In the first "improved" example, I have no way of knowing what the columns meaning without them.
The only good improvement in this is the "#" convention as a comment. If we could get every CSV parser in future to agree (we already can't get them to agree now) to ignore lines that start with "#" then CSV's would be vastly improved.
I can't see this gaining any traction. There's nothing wrong with CSV as it is for basic data transfer. CSV had been unchanged now for so long, that any attempt to update it will fail - there are simply too many CSV parsers already implemented. If you are worried about people in the year 3000 understanding your data, add a .txt file alongside the CSV explaining the fields.
I disagree. CSV is horribly underspecified and many parsers have conflicting ideas on how things should work. I've hit areas where, for example, an API was generating a CSV that another API could read, but NOT if it was opened and then re-saved by Excel first (just re-saving, not editing), because Excel was making some trivial change in the format that the consuming parser couldn't handle. Then there's issues with encoding, localisation, and escaping (see, eg, https://chriswarrick.com/blog/2017/04/07/csv-is-not-a-standa... although that's just scratching the surface).
That being said, this proposal seems worse in every way that existing CSVs, but I don't think the issue is "there's a well specified CSV standard with tons of parsers that won't change to support this new one", it's more that there is no standard, all the parsers are incompatible, and no meaningful improvements are possible. :(
I can't see this gaining any traction. There's nothing wrong with CSV as it is for basic data transfer. CSV had been unchanged now for so long, that any attempt to update it will fail - there are simply too many CSV parsers already implemented. If you are worried about people in the year 3000 understanding your data, add a .txt file alongside the CSV explaining the fields.
> There's nothing wrong with CSV as it is for basic data transfer. I disagree. CSV is horribly underspecified and many parsers have conflicting ideas on how things should work. I've hit areas where, for example, an API was generating a CSV that another API could read, but NOT if it was opened and then re-saved by Excel first (just re-saving, not editing), because Excel was making some trivial change in the format tha…
I can't see this gaining any traction. There's nothing wrong with CSV as it is for basic data transfer. CSV had been unchanged now for so long, that any attempt to update it will fail - there are simply too many CSV parsers already implemented. If you are worried about people in the year 3000 understanding your data, add a .txt file alongside the CSV explaining the fields.
And while CSV is primarily used for data transfer, Excel could hypothetically be a great debug-viewer if it didn't do brain dead stuff like unrecoverably corrupt UPCs and other long numbers by default.
There's an Excel Uservoice about this very issue from 2015:
https://excel.uservoice.com/forums/304921-excel-for-windows-...
They're "looking into it." Just as they've been looking into it for the last twenty years. Any. Day. Now.
I can't see this gaining any traction. There's nothing wrong with CSV as it is for basic data transfer. CSV had been unchanged now for so long, that any attempt to update it will fail - there are simply too many CSV parsers already implemented. If you are worried about people in the year 3000 understanding your data, add a .txt file alongside the CSV explaining the fields.
I agree. The biggest problem with CSV is Excel's terrible support. And while CSV is primarily used for data transfer, Excel could hypothetically be a great debug-viewer if it didn't do brain dead stuff like unrecoverably corrupt UPCs and other long numbers by default. There's an Excel Uservoice about this very issue from 2015: https://excel.uservoice.com/forums/304921-excel-for-windows-... They're "looking into it."…
How do you define headers? There's no header for the bottom csv version so I have to manually type headers for my data frame?
I got a dataset with 87 obs and ~8700 columns here. I'm not going to manually name those columns. What's the solution?
Earlier quoted context omitted.
> There's nothing wrong with CSV as it is for basic data transfer. I disagree. CSV is horribly underspecified and many parsers have conflicting ideas on how things should work. I've hit areas where, for example, an API was generating a CSV that another API could read, but NOT if it was opened and then re-saved by Excel first (just re-saving, not editing), because Excel was making some trivial change in the format tha…
To your point it's hard to see how the referenced article qualifies as a standard. It does not deal with escape characters or missing values, two places where CSV implementations tend to vary in random ways.