So, which library? CSV is a mess.
ocaml-csv can handle anything Excel can throw at it (and throw it back). Worked out well in the real world.
So You Want to Write Your Own CSV code
81–90 of 127 posts
Re: So You Want to Write Your Own CSV code
#82So, which library? CSV is a mess.
Have you tried Papa Parse?
Re: So You Want to Write Your Own CSV code
#83We actually use CSV-reading as an incidental part of a hiring exercise. We provide a really simple homemade CSV parser as part of a PHP project, with a "could you find and fix bugs in this?" instruction. The way to get full marks is to rip out the parser and replace it with the appropriate standard library function.
I like this.
Only thing that I don't like is that many candidates will assume that they have to fix the code within the parser, given those instructions, even if they know that a battle-tested library is how they would actually do it. I hope you accept an off-hand comment such as, "ew, why is this hand-rolled" as a sufficient indicator in favor of your solution.
Re: So You Want to Write Your Own CSV code
#84CSV are a headache. Like the article says, RFC4180 doesn't necessarily represent the real world. However sometimes you just have to reject things that aren't spec. Not too long ago I was struggling with one of these CSV issues and received some good advice from Hans Passant [1] on a Stack Overflow question pertaining to my problem (emphasis mine): "It is pretty important that you don't try to fix it. That will make y…
Yeah, that's a great idea if you can.
I was pulling data from a medical system that I knew full well I would not be able to get changes into for YEARS (and I got to meet the vendor, who was working on a shiny new XML export system - I wonder if that has quoting issues too - it wasn't released by the time I finished working the project)
So I wound up writing perl that knew enough to fix all the common problems with the source data, and emailed me any odd lines it couldn't cope with, so I could go in and update the regular expressions. It kinda sucked, but the end result was better antibiotic coverage for a bunch of people. Worst case of a line it really couldn't handle was that person didn't get the benefit of an expert system checking that they didn't have doubled-up medicines, which is no worse than they would have had without this system.
Re: So You Want to Write Your Own CSV code
#85Post a sample .csv file you believe is too difficult. I will solve your problem with only UNIX utilities. And I'm sure others will solve it other ways. Usually I only need sed and tr. Sometimes lex or AWK. Arguing about something without ever pointing to an example accomplishes nothing; it's just whining. Post an example. Thank you.
It isn't that any particular file is difficult but that the variations that you haven't even thought about might catch you out. It is the deceptive simplicity of the samples that you have at hand that may catch out your code when it hits a different (also simple but different) example in the field.
Re: So You Want to Write Your Own CSV code
#86Its a flippin' CSV.
Of course you can come up with scenarios where it doesn't work, but anyone who considers themselves to be a competent programmer should be able to deal with these issues, use another data format, or just talk to whomever is giving you the data to correct their data issues.
Seriously, The overhwleming CSV_bashing in these comment really makes me worry that coders just can't handle the basics anymore.
Re: So You Want to Write Your Own CSV code
#87Its a flippin' CSV. Of course you can come up with scenarios where it doesn't work, but anyone who considers themselves to be a competent programmer should be able to deal with these issues, use another data format, or just talk to whomever is giving you the data to correct their data issues. Seriously, The overhwleming CSV_bashing in these comment really makes me worry that coders just can't handle the basics anymor…
It's not a question of can, it's a question of should. If any engineer on my team came to me and told me he was building a CSV reader/writer, I would seriously question his judgement as an engineer[1]. My thoughts would be that either he isn't capable of seeing obvious challenges in building a "simple" CSV feature or he isn't able to prioritize his time well, focusing on useless toys at the expense of getting important work done.
1. Of course there are exceptions to the rules: perhaps the CSV is malformed or there are special considerations in the backend, but the general point stands.
Re: So You Want to Write Your Own CSV code
#88A good and performant alternative to CSVs are Google's protocol buffers: https://code.google.com/p/protobuf/
Re: So You Want to Write Your Own CSV code
#89Earlier quoted context omitted.
In General Swedish csv files are separated with ; I have an function that I usually use in projects that counts , and ; on each line to determine which one is most likly beeing used in the file. The most annoying thing I have found in csv files is the escape sign I would like it to be \" but very often I see """ as the escape for "
I believe that the Swedish Klarna files uses ; but the Danish ones uses the comma + space. That only adds to the stupidity though, why not have just one format?
Just be happy to live in the best country.
Re: So You Want to Write Your Own CSV code
#90Why are people using CSV when better (and less fuzzily defined) solutions exist, such as JSON?
CSV is far, far more ubiquitous and much more usable in non-web settings. (e.g. desktop data analysis programs)
true, and in those settings you largely don't see situations that trip up naive parsers such as newlines or delimiters inside fields.