Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Time to retire the CSV?

bitsondisk.com

Post reply on HN
Print view View original

51–60 of 594 posts

Re: Time to retire the CSV?

#51

Post by _jal »

The more interesting question is to try to tease apart what led to CSV's survival, despite all the warts. Clearly a combination of things, including accident. But I'll point out some advantages:

- Repairability by mere mortals. If you are competent enough to use Excel and are given bad data, you are competent enough to fix it. (Whether the dataset is too big for mere mortals to find the problem is a different issue.)

- Trivial serialization. It is super-easy to dump things to CSV. (This is probably also why there are so many annoying variants.)

- Usable by other tools. You don't need special libraries to read it, so pipelines involving the usual suspects is possible. Importantly, there are a ton of different ways to do this sort of thing, so non-experts can frequently find something that works for them, even if it looks wonky to programmers.

All that said, I hate dealing with them, too.

Re: Time to retire the CSV?

#52

Post by NBJack »

MeteorMarc wrote: post #18
coding123 wrote: post #9

As long as it's replacement opens in Numbers and Excel.

CSV does not open in Excel, unless you play the usual tricks first...

Not sure what your basing that claim on. It is a natively supported format, even amidst its inconsistencies, and has been for decades.

Are you referring to a particular format variant?

Re: Time to retire the CSV?

#53

Post by Twisell »

Sunspark wrote: post #23

I don't agree with giving up csvs until the following conditions are met: 1) A truly open format is available and accessible. Csvs are textfiles. There is no system around that cannot open a textfile. If the format is binary or requires patents or whatever, then it's a non-starter. 2) Applications have a speed increase from using csvs. To wit, I loved csvs because often they finish preparing much faster than a "forma…

OP and you gave me an idea :

"The only true successor of CSV should be forward/backward compatible with any existing CSV variant"

If we manage to write a spec that meet this criteria we'll have a powerful standard with easy adoption.

Re: Time to retire the CSV?

#54

Post by dullgiulio »

nolok wrote: post #24
podiki wrote: post #16

As long as it is text-based. At least you can actually look at a CSV and see what is going (wrong), as well as use all the text tools we have. Not that there aren't problems as the article points out. But some binary file based on the whims of a proprietary program...no thanks.

Xlsx (office format for almost a decade now) are zip and XML all the way. Not fun to look at, but totally readable by a human.

Readable but not comprehensible. CSV is hard to beat in that sense, being somewhat "natural" like a table written on paper.

Re: Time to retire the CSV?

#55

Post by masklinn »

Sanzig wrote: post #44
masklinn wrote: post #17

Earlier quoted context omitted.

And CSV can be consumed and produced by spreadsheet software, which is useful to critical in many contexts where CSV is part of the pipeline. There the alternative would be to use xlsx files which… isn't necessarily any better. Until someone gets excel to ingest and produce something in a better format, we're pretty much stuck.

As I mentioned down-thread, I can generate a CSV with a couple of fprintf statements and a loop. I definitely can't do that with .xlsx. There is almost zero friction to bolting CSV export capability to an existing system, which is part of why it's so popular.

> As I mentioned down-thread, I can generate a CSV with a couple of fprintf statements and a loop.

And usually generate garbage for anything but the most trivial case, which really nobody gives a shit about. That's the main reason why CSV absolutely sucks too, you have to waste month diagnosing the broken shit you're given to implement the workarounds necessary to deal with it.

> I definitely can't do that with .xlsx.

You probably can though. An xlsx file is just a bunch of XML files in a zip.

Re: Time to retire the CSV?

#56

Post by habitue »

wutwutwutwut wrote: post #41
habitue wrote: post #7

Earlier quoted context omitted.

Objectively, CSV is terrible for humans despite being a plaintext format. No one reads CSVs: they're incomprehensible since the columns are not aligned with the headings. (You might be drawing an analogy with JSON, which is often human readable because it puts the keys right there next to the values). The best that can be said for its simplicity is that it's easy to write code that can dump data out in CSV format (an…

Which format would you propose if I am writing a .NET tool to produce a file to be consumed by Excel?

I would expect .NET to have excellent .xlsx capabilities? Most libraries have an easy way to write those from tabular data.

Re: Time to retire the CSV?

#57

Post by rutthenut »

AdmiralAsshat wrote: post #38

CSV was a thing long before I was born, so I'm not privy to how it came about. But at least in day-to-day work, the single biggest drawback of CSV in my experience is the fact that the comma and most of the other common delimiters occur regularly in real data, forcing all of the cumbersome escape sequences. To say nothing of someone misplacing a quote somewhere and throwing off the cell count. So, question to the gre…

Agree that the commma delimiter is a pain, as it is a frequently used character in the content. Then adding escape-type chars or quotes around fields just makes things more and more messy.

Consider also that IBM EBDCIC and other non-ASCII character sets were (and are) in common use, so C0/C1 may not have made sense 'back in the day'

Re: Time to retire the CSV?

#58

Post by hestefisk »

20 years ago, clever consultants said the same when XML was the “next big thing”. I remember reading this book called “The XML Revolution”, which boldly stated that the age of CSV and other “obscure” formats was finally over. Here we are 20 years later with the same rhetoric. CSV isn’t pretty, no, but it’s universally acceptable and works quite nicely on a Unix command line with cat, grep, and cut, so I am sure it will stick around for longer.

Re: Time to retire the CSV?

#59

Post by daxuak »

nolok wrote: post #24
podiki wrote: post #16

As long as it is text-based. At least you can actually look at a CSV and see what is going (wrong), as well as use all the text tools we have. Not that there aren't problems as the article points out. But some binary file based on the whims of a proprietary program...no thanks.

Xlsx (office format for almost a decade now) are zip and XML all the way. Not fun to look at, but totally readable by a human.

Human can't efficiently write or parse XML or json, though. In some scenarios CSV hits the right spot to be accessible to human and computer, and the table can be laid out so that one can sort/grep/awk to quickly gain some insight.

Re: Time to retire the CSV?

#60

Post by puppet-master »

MonadIsPronad wrote: post #10

Surely the successor should be SQLite. Tabular data like CSV, easy to view on different mediums, free and open source, single file databases. It has to be this, right?

Replacing an almost trivial open format with one that requires a specific 85 kLOC codebase to access does not seem like an improvement.

SQLite is wonderful, but it's definitely not a replacement for CSV

Post reply on HN