51–60 of 127 posts
Why are people using CSV when better (and less fuzzily defined) solutions exist, such as JSON?
No alternate delimiters, no backslashes.
Now I have to put up with offshore staff trying to use apostrophes (') instead of quotes (") :-(
Barring alternate delimiters, and disallowing newlines* in fields, I can write the parser for 4180 in about 30 lines of perl, reading a char at a time and flipping between about 4 states. (avoids getting root access and days of paperwork to install from CPAN)
* disallowing newlines in the data is admittedly a big restriction, but it works for many use-case/applications, and allows the caller to pull in a line before calling the parse function.
For Java, the "Ostermiller" library is pretty good for CSV handling, and has a few options for dealing with freaky variants.
Earlier quoted context omitted.
True, but in my mind picking ", " indicate to me that they don't care or don't know what they're doing. I often run into something similar with XML. I've had more than one partner call or write me saying that the elements in a file are not in the right order. Every single time they've admitted to not actually using an XML parser. Don't do things that screw up the standard tools other developers depend on.
Stuff not being in the correct order is a perfectly fine technical reason to reject an XML file if you have a DTD based workflow. It is actually quite difficult to specify that the order of elements is irrelevant (it goes with n!, so allowing six elements exactly once, but in arbitrary order in the contents of So in that case, you might have been the developer that screwed up the standard tools other developers depen…
CSV are a headache. Like the article says, RFC4180 doesn't necessarily represent the real world. However sometimes you just have to reject things that aren't spec. Not too long ago I was struggling with one of these CSV issues and received some good advice from Hans Passant [1] on a Stack Overflow question pertaining to my problem (emphasis mine): "It is pretty important that you don't try to fix it. That will make y…
In my current job, the most common "invalid" CSV format we get is .xlsx files.
So I wrote an .xlsx parser (way, way faster than Apache POI).
Another interesting hiccup to consider is CSV inside individual fields - i.e. recursive CSV. There are various ways to handle this, but in my company's line of business the usual route is to duplicate that line once per CSV element found in the field.
Likely the next invalid format we'll have to parse is PDFs containing tables...
as the article mentions, CSV is not well defined. libraries are.. well, different. you'd spend as much time becoming familiar with one as you would writing a basic parser. commas don't delimit field entries? CSV -> comma separated values. new lines inside a field? i've never written a parser that would be foiled by this. could be an issue if you use a built-in tokeniser (e.g strtok, etc.). be aware. variable number o…
Too many "enterprise" coworkers who don't know how to write a finite state machine. They do need to use a library.
CSV are a headache. Like the article says, RFC4180 doesn't necessarily represent the real world. However sometimes you just have to reject things that aren't spec. Not too long ago I was struggling with one of these CSV issues and received some good advice from Hans Passant [1] on a Stack Overflow question pertaining to my problem (emphasis mine): "It is pretty important that you don't try to fix it. That will make y…
> Not kidding.
We'd ll be better off really, but that ship has sailed. Using CSV for data which is only ever read by a machine is a dumb decision. Use the RS (record separator) character and many of these ambiguities disappear.
Of course, like I said, that ship has sailed. If you want your data to be read nicely by other programs you're probably stuck with CSV, TSV, or something similar.
I can't but disagree when I read stuff like this. Why shouldn't I release a library if I think it's good enough for the community? Even the powerful and versatile Ruby library for CSV parsing started as a gem from a person who didn't give a s... about advise like "do not inflict another one into this world".