Time to retire the CSV?
bitsondisk.com
Time to retire the CSV?
1–10 of 594 posts
Re: Time to retire the CSV?
#2“ I’m not going to advocate for any one successor file format here, but there are certainly a lot of contenders. The big data ecosystem gave us formats like Avro, Parquet, and Arrow that are widely used as an intermediate representation when transferring data between systems. HDF51 is widely used in the scientific computing community.”
Xkcd: https://m.xkcd.com/927/
Re: Time to retire the CSV?
#3"In favour of what?", that is the matter.
CSV is a format more for humans and less for machines, but that is the use case: a format that is good enough to be compiled by humans and read by machines. At the moment there aren't many alternatives.
Re: Time to retire the CSV?
#4It's probably not news to anyone who works with data that CSVs are inferior to other file formats like parquet.
I think the valuable insight here is that there needs to be a meme / movement that CSV is bad or deprecated. That's what's actually going to put the nails in its coffin, not private griping from developers when they get CSVs.
I'm all for it. Down with CSV :)
Re: Time to retire the CSV?
#5I agree with many of the problems - but luckily many providers (of data) use the first lines to provide a header of metadata.
Re: Time to retire the CSV?
#6“ I’m not going to advocate for any one successor file format here, but there are certainly a lot of contenders. The big data ecosystem gave us formats like Avro, Parquet, and Arrow that are widely used as an intermediate representation when transferring data between systems. HDF51 is widely used in the scientific computing community.” Xkcd: https://m.xkcd.com/927/
USB-C (alt text)
Re: Time to retire the CSV?
#7"In favour of what?", that is the matter. CSV is a format more for humans and less for machines, but that is the use case: a format that is good enough to be compiled by humans and read by machines. At the moment there aren't many alternatives.
Objectively, CSV is terrible for humans despite being a plaintext format. No one reads CSVs: they're incomprehensible since the columns are not aligned with the headings. (You might be drawing an analogy with JSON, which is often human readable because it puts the keys right there next to the values).
The best that can be said for its simplicity is that it's easy to write code that can dump data out in CSV format (and to a lesser extent, it's easy to parse it, though watch out for those variants). This is not a really strong argument, most everyone is going to use a library for serialization, there's no reason to write your own unless it's for learning.
Re: Time to retire the CSV?
#8"In favour of what?", that is the matter. CSV is a format more for humans and less for machines, but that is the use case: a format that is good enough to be compiled by humans and read by machines. At the moment there aren't many alternatives.
I think this issue can be bypassed if a better format such as Arrow or Parquet can be used in Excel.
Re: Time to retire the CSV?
#9As long as it's replacement opens in Numbers and Excel.
Re: Time to retire the CSV?
#10Surely the successor should be SQLite. Tabular data like CSV, easy to view on different mediums, free and open source, single file databases. It has to be this, right?