CSV is perfectly fine for bring data from one App to another App. Is is easy. Everybody can fix things with any editor. You can paste files together, you can seperate files after x lines. Of course not everything is perfect .. and Microsoft made it even worser in the german office. The seperator is not even "," it is ";". The author writes about problems with dates .. i think we everybody saw excel translate dates to funny floats and so on. His problem with the date format is not a csv problem. The official standart is just not accepted in dayli life. Americans have anyway the most terrible ways to write dates. There is everything lost ..
CSV is almost perfect for the usecase to bring data from one app to another without trouble.
Time to retire the CSV?
371–380 of 594 posts
Re: Time to retire the CSV?
#372Earlier quoted context omitted.
> access through cat/grep/awk, and easily load into any programming language Until the CSV fields contain commas themselves. Even if fields are surrounded by "".
I recall one of my old Phones could dump SMS messages as CSV. I wanted to be build and interface that would let me search through my messages. Turns out when you have data that contains both commas and quotes things get screwey real quick. You could have quoted data like: 123,ABC,”,””,456 Where ,” is column data. I think that standard method is to double quote the field, but the dump sure wasn’t doing that for me, pl…
[deleted]
Re: Time to retire the CSV?
#373Earlier quoted context omitted.
You're comparing CSVs to other spreadsheet document formats. But a CSV is not a spreadsheet. A CSV is raw data. (It's data that is restricted to a shape that enables it to be easily imported into a spreadsheet—but data nevertheless.) As such, it should be compared to other data formats—e.g. YAML, JSON Lines, etc. These other data formats all win on your #2 against CSV, as CSV is actually horrible at parse-time vs. ot…
>the fact that both of its separators (newlines and commas) can appear as-is inside column values, with a different meaning, if those column-values are quoted, means that there's no way to parallelize CSV processing, because there's no way to read-ahead and "chunk" a CSV purely lexically Yes, this is a major pain. It can be avoided by using Tab separated value (TSV) files, which don't use escaping. But then you can't…
ASCII has special delimiters 0x1E Record Separator and 0x1F Unit Separator to avoid conflicting with values, but they have never gained widespread adoption.
Re: Time to retire the CSV?
#374Earlier quoted context omitted.
Take an Excel file and change the extension to .zip, then extract the contents. You will see that it is a collection of XML files. Therefore it should be reasonable to conclude that this approach can work for Excel sized datasets. However it is not particularly readable/diff-able if this is part of your use case.
Excel actually defines a simple XML-based alternative to CSV: https://en.wikipedia.org/wiki/Microsoft_Excel#XML_Spreadshee...
[deleted]
Re: Time to retire the CSV?
#375This reads like a joke. If you think you can do better than CSV, let's see your proposal. Hint: you probably can't, and if you could, you probably couldn't get Excel to export it, so you still probably can't. "The status quo is bad, more recent popular formats aren't good enough either, but I don't actually have a specific proposal that's better than all of the above" is a lot faster to read than that article, and sa…
Just another confirmation for Betteridge's Law of Headlines
Re: Time to retire the CSV?
#376Earlier quoted context omitted.
How is a csv not the most accurate representation of the data? If you trust the other agent encoded it properly in the db, then sure. Your flippant dismissal was inappropriate in tone and detracted from the rest of your opinion. Cockiness tells me that you’re insecure about your knowledge, not that you know more than GP.
> How is a csv not the most accurate representation of the data? If you trust the other agent encoded it properly in the db, then sure. The idea that a CSV would be more likely to be correctly encoded than a DB is hilarious, thanks for the laugh. But that you were confident enough to seriously put it in writing shows how little experience you have with CSV.
Yep, you got me. I'm actually a trapeze artist moonlighting on these forums.
A CSV file represents the exact digits that are to be stored. You have unlimited precision. You could even store irrational numbers, equations, or mix data types in a column. OTOH, you have to make sure the delimiting character is not present in the data - that can be pretty easy, if you use the field-separator character in ASCII, or even just a \t. I've even seen people terminate fields with ^8675309| because they felt confident no data would contain Jenny's number.
A database, like Excel, likes to conform data. This is usually awesome! But sometimes, it's not.
Re: Time to retire the CSV?
#377> It's Time to Retire the CSV > This column obviously contains dates, but which dates? Most of the world It's time to retire local formats and always write YYYY-MM-DD (which is both the international and the Swedish standard, and the most convenient for parsing and sorting). > A third major piece of metadata missing from CSVs is information about the file’s character encoding. It's bloody the time to retire all the c…
CSV is accessible to programmers and non-programmers alike. HDF5 and SQLite require some programming ability and special software to interact with the data as they are not just text files like CSV.
Re: Time to retire the CSV?
#378Earlier quoted context omitted.
How is a csv not the most accurate representation of the data? If you trust the other agent encoded it properly in the db, then sure. Your flippant dismissal was inappropriate in tone and detracted from the rest of your opinion. Cockiness tells me that you’re insecure about your knowledge, not that you know more than GP.
There's plenty of CSVs that have been produced or will be parsed by for line in input: ','.join(line) It's not exactly a problem with "CSV" specifically, but the environment in which it exists.
True, but if you don't trust the code generating the data, then any and every data format is suspect.
Re: Time to retire the CSV?
#379Earlier quoted context omitted.
Thats just a CSV with extra steps.
Nope, that's CSV without the drawbacks of CSV. That's CSV that can have special characters and doesn't suffer from delimiter problems. When someone says "Maybe we can fix CSV" this is what you should do instead of trying to "fix" CSV.
[deleted]
Re: Time to retire the CSV?
#380Earlier quoted context omitted.
You're comparing CSVs to other spreadsheet document formats. But a CSV is not a spreadsheet. A CSV is raw data. (It's data that is restricted to a shape that enables it to be easily imported into a spreadsheet—but data nevertheless.) As such, it should be compared to other data formats—e.g. YAML, JSON Lines, etc. These other data formats all win on your #2 against CSV, as CSV is actually horrible at parse-time vs. ot…
>CSV is actually horrible at parse-time vs. other data formats I find this really hard to believe given it's a simple enough syntax. And parsing is usually not the limiting factor, usually fast enough to not be noticed alongside interpreting or loading the source data. Every (much more sophisticated) compiler I can think of uses a linear parser based on this assumption.
Just extracting string views can be fast, but converting numbers to and from text is very slow.