681–690 of 711 posts
Earlier quoted context omitted.
The practical solution is to generate several CSV files and distribute work at the granularity of files
Sure, now you need to do this statically ahead of time. It's not unsolvable, but now you have a more complicated system. A better file format would not have this problem. The fix is also trivial (escape new lines into \n or similar) would also make the files easier to view with a text editor.
Earlier quoted context omitted.
Not sure if they still do this, but Klarna would send us ", " separated files. If there wasn't a space after the comma then it was to be read as a decimal point. Most of the CSV parser don't/didn't allow you to specify multi-character separators. In the end I just accepted that we had one field for krona and for öre and most fields would need to have a leading space removed.
Microsoft did this very extensively. Many Non-English versions of Excel do save CSV-files with a semicolon as a separator and it probably was handled differently too in normal Excel files. But it goes even further, it affected their scripting languages even to this day with newer languages like their BI script (forgot the name of the language). For example, parameters of function calls aren't separated by ',' anymore…
Many European languages use '.' to end statements, Prolog (France) for example, but use ';' to separate arguments.
I also love CSV for its simplicity. A key part of that love is that it comes from the perspective of me as a programmer . Many of the criticisms of CSV I'm reading here boil down to something like: CSV has no authoritative standard, and everyone implements it differently, which makes it bad as a data interchange format. I agree with those criticisms when I imagine them from the perspective of a user who is not also a…
> I know I can quickly write a parser, not by reading some spec, but by looking at the actual CSV file This is fine if you can hand-check all the data, or if you are okay if two offsetting errors happen to corrupt a portion of the data without affecting all of it. Also I find it odd that you call it "easy" to write custom code to parse CSV files and translate between CSV formats. If somebody give you a JSON file that…
But the premise of CSV is so simple, that there are only four quirks to empirically discover: cell delimiter, row delimiter, quote, escaped-quote.
I think it's "easy" to peek at the file and say, "Oh, they use semicolon cell delimiters."
And it's likewise "easy" to write the "custom logic", which is about as simple as parsing something directly from a text stream gets. I typically have to stop and think a minute about the quoting, but it's not that bad.
If a programmer is practiced at parsing from a text stream (a powerful, general skill that is worth exercising), than I think it is reasonable to think they might find parsing CSV by hand to be easier and quicker than parsing JSON (etc.) with a library.
I so hate CSV. I am on the receiving end: I have to parse CSV generated by various (very expensive, very complicated) eCAD software packages. And it's often garbage. Those expensive software packages trip on things like escaping quotes. There is no way to recover a CSV line that has an unescaped double quote. I can't point to a strict spec and say "you are doing this wrong", because there is no strict spec. Then ther…
Try to live in a country where "," is the decimal point. Of course this causes numerous interoperability issues or hidden mistakes in various data sets. There would have been many better separators... but good idea to bring formatting into it as well...
Earlier quoted context omitted.
Yes but in practice CSV is defined by what Excel does. As there is no standard to which Excel conforms as it predates standards and there would be an outcry if Excel started rejecting files that had worked for years.
There is a common misconception here. You can import CSV files into an excel sheet. You cannot open a CSV file with excel. That is a nonsense operation.
CSV is bad. Furthermore it’s unnecessary. ASCII has field and record separator characters that were for this purpose.
That would be great if keyboards had keys for those characters and there was a common way to display them on a screen, but they don't and there isn't.
Earlier quoted context omitted.
Agreed. Which means that Javascript does not have a good parser.
`JSON.parse` actually does give you that option via the `reviver` parameter, which gives you access to the original string of digits (to pass to `BigInt` or the number type of your choosing) – so per this conversation fits the "good parser" criteria.
https://caniuse.com/mdn-javascript_builtins_json_parse_reviv...
Absent in Safari though
Earlier quoted context omitted.
Graphical representations of the control characters begin at U+2400 in the "Control Pictures" Unicode block. Instead of the actual U+001E Record Separator, you put the U+241E Symbol for Record Separator in the help text.
.... with a note underneath urging readers not to copy and paste the character because it's only the graphical representation of it, not the thing itself. Perhaps a more salient example might be CSV nested in CSV. This happens all the time with XML (hello junit) and even JSON— when you plug a USB drive into my LG TV, it creates a metadata file on it that contains {"INFO":"{ \"thing\": true, }"}
Earlier quoted context omitted.
If you disallow control characters so that you can use them as delimiters, then CSV itself becomes a "binary" data format - or to put it another way, you lose the ability to nest CSV. It isn't good enough to say "but people don't/won't/shouldn't do that", because it will just happen regardless. I've seen nested CSV in real-life data. Compare to the zero-terminated strings used by C, one legacy of which is that Postgr…
So have a way to escape those control characters.