Live data from Hacker News

So You Want to Write Your Own CSV code

tburette.github.io

71–80 of 127 posts

Re: So You Want to Write Your Own CSV code

#71
post #30
post #27

Earlier quoted context omitted.

it can be irritating, but you can just as easy parse ", " to "|" or something, by simple string replacing, pre parsing..

Think it through. What if there is free text in the field? "How are you, Sally?"

If your delimiter appears in a field, the field needs to be quoted.

Re: So You Want to Write Your Own CSV code

#72
post #60

"CSV is not a well defined file-format. The RFC4180 does not represent reality. It seems as every program handles CSV in subtly different ways. Please do not inflict another one onto this world. Use a solid library." I can't but disagree when I read stuff like this. Why shouldn't I release a library if I think it's good enough for the community? Even the powerful and versatile Ruby library for CSV parsing started as…

IF your library is a solid library, then release it. What he is saying though, is don't roll your own if you can use a solid library. And if a good solid library exists, why bother writing your own?

Re: So You Want to Write Your Own CSV code

#73

The most retard structure I've seen in a CSV file relates to the "What if the character separating fields is not a comma?". We get "CSV" files from Klarna, an invoicing company, with the payments they've processed for us. Because we're Danish and they are Swedish, it's not really weird that they would use comma as the decimal separator. So to compensate for having used the comma, they for some reason picks ", " ( tha…

In General Swedish csv files are separated with ; I have an function that I usually use in projects that counts , and ; on each line to determine which one is most likly beeing used in the file. The most annoying thing I have found in csv files is the escape sign I would like it to be \" but very often I see """ as the escape for "

Well the standard says to double the double quote to escape it. If you are seeing \" or even """ to escape a quote, then someone isn't following the standard.

Re: So You Want to Write Your Own CSV code

#74
post #61

CSV works for simple cases. It is trivial to parse, you shouldn't even need a library. It there are many "what ifs" like in the posted article. You probably need another format like JSON (preferably) or XML.

Well a lot of times you are reading in a csv file given to you by someone else. You can't expect it to be "simple" and you need to parse it correctly

Re: So You Want to Write Your Own CSV code

#75

This article makes it much more complicated than it needs to be. It tries to be all things to all people. In practice you're going to have to sacrifice some functionality for the sake of usability and your own sanity. When I add a CSV import feature to a project I'm working on, I tell people "this works with MS Excel flavor of CSV." This covers most, if not all, real world cases because in my world the people who wan…

What is wrong with trying to be all things to all people? If you use a good solid library you don't need to tell people "this works with some versions of MS Excel" And that is the main point of the article.

Re: So You Want to Write Your Own CSV code

#76
Post a sample .csv file you believe is too difficult.

I will solve your problem with only UNIX utilities. And I'm sure others will solve it other ways.

Usually I only need sed and tr. Sometimes lex or AWK.

Arguing about something without ever pointing to an example accomplishes nothing; it's just whining.

Post an example.

Thank you.

Re: So You Want to Write Your Own CSV code

#77
post #6

Why are people using CSV when better (and less fuzzily defined) solutions exist, such as JSON?

Because Excel doesn't export JSON. Because if you have a table, offering comma delimited field is easy. But really, people you work with give you csv files, and you don't have a choice.

Re: So You Want to Write Your Own CSV code

#79
post #36

CSV are a headache. Like the article says, RFC4180 doesn't necessarily represent the real world. However sometimes you just have to reject things that aren't spec. Not too long ago I was struggling with one of these CSV issues and received some good advice from Hans Passant [1] on a Stack Overflow question pertaining to my problem (emphasis mine): "It is pretty important that you don't try to fix it. That will make y…

This is good advice. It's the philosophy adopted by Papa Parse, http://papaparse.com - try to gracefully handle malformed CSV and report all errors so they're obvious and actually helpful by telling the user where the syntax error is at.

Trying to compensate for bad CSV format will more likely cause headaches and confusion rather than clarity. It can also discourage the need for CSV writers to be rigorous about their output formatting.

Re: So You Want to Write Your Own CSV code

#80
post #61

CSV works for simple cases. It is trivial to parse, you shouldn't even need a library. It there are many "what ifs" like in the posted article. You probably need another format like JSON (preferably) or XML.

JSON is arguably more complex than CSV. Though it is at least well-defined (mostly).

However, that doesn't excuse sloppy CSV writers.

Post reply on HN