The reason why USV did not use the proper ASCII codes for field separator and record separator is a bit too pragmatic for me…
https://github.com/SixArm/usv/tree/main/doc/faq#why-use-cont...
Friends don't let friends export to CSV
241–250 of 459 posts
Re: Friends don't let friends export to CSV
#242Earlier quoted context omitted.
Totally agree. His arguments are basically "performance!" (which is honestly not important to 99% of CSV export users) and "It's underderspecified!" And while I can agree with the second, at least partly, in the real world the spec is essentially "Can you import it to Excel?". I'm amazed at how much programmers can discount "It already works pretty much everywhere" for the sake of more esoteric improvements. All that…
ASCII has had field and record separators since like, forever. Wish we had kept using those.
Re: Friends don't let friends export to CSV
#243Earlier quoted context omitted.
ASCII has had field and record separators since like, forever. Wish we had kept using those.
Does it have nesting operators? I want to embedd ASCII within my ASCII fields. So I can have a table within my table.
It kind of does. See `man ascii`
* FS (0x1C) file separator
* GS (0x1D) group separator
* RS (0x1E) record separator
* US (0x1F) unit separator
I've never seen these in the wild though.
Re: Friends don't let friends export to CSV
#244Re: Friends don't let friends export to CSV
#245As a French, there is another problem with CSV. In the French locale, the decimal point is the comma, so "121.5" is written "121,5". It means, of course, that the comma can't be used as a separator, so the semicolon is used instead. It means that depending whether or not the tool that exports the CSV is localized or not, you get commas or you get semicolons. If you are lucky, the tool that imports it speaks the same…
Does any tool seriously localize CSVs?
Re: Friends don't let friends export to CSV
#246excel 2021:
the "a spreadsheet is all it needs"-file is not usable because excel is not able to translate the "LC references that are inside brackets" into other languages.
Re: Friends don't let friends export to CSV
#247Earlier quoted context omitted.
I can't remember the last time I, or anyone I've ever worked with for that matter, ever typed up a CSV from scratch. The whole point of USV is that the delimiters can't normally be typed so you don't have to worry about escaping. USV supports displayable delimiters (see https://github.com/SixArm/usv ), so for the much more common case of editing an existing CSV in a text editor, you can just copy and paste.
Everyone of us was a beginner at some point. The first time we came across CSV format we likely typed it in notepad by hand. A lot of issues with CSVs are also sometimes troubleshooted by hand-- by manually fixing a quote or a comma. There is value is the ability to do this level of editing and troubleshooting.
No we didn't, we likely typed in Excel by double clicking on our first csv
Re: Friends don't let friends export to CSV
#248Earlier quoted context omitted.
> In the French locale, the decimal point is the comma, so "121.5" is written "121,5". It means, of course, that the comma can't be used as a separator Heh. Ah, HN. Always good for a laugh line.
Yeah, hon hon hon and all, but one of my (US) bank statements exports a CSV which uses commas in numbers in the US fashion, so $1,500 and the like. Writing a custom CSV munger to intake that into ledger-csv was... fun, but then again, only had to do it once.
[deleted]
Re: Friends don't let friends export to CSV
#249Earlier quoted context omitted.
Totally agree. His arguments are basically "performance!" (which is honestly not important to 99% of CSV export users) and "It's underderspecified!" And while I can agree with the second, at least partly, in the real world the spec is essentially "Can you import it to Excel?". I'm amazed at how much programmers can discount "It already works pretty much everywhere" for the sake of more esoteric improvements. All that…
ASCII has had field and record separators since like, forever. Wish we had kept using those.
No you don’t. It’s a holdover from when files were on tapes. The logic is all inverted too. Record separators were at the beginning of a record. Group and unit separators could then be nested. You really needed a spec for whatever document you were trying to makt
Re: Friends don't let friends export to CSV
#250The reason CSV is popular is because it is (1) super simple, and (2) the simplicity leads to ubiquity. It is extremely easy to add CSV export and import capability to a data tool, and that has come to mean that there are no data tools that don't support CSV format. Parquet is the opposite of simple. Even when good libraries are available (which it usually isn't), it is painful to read a Parquet file. Try reading a Pa…
> But Protobuf is not a splitable format, which means it is not Big Data friendly, unlike Parquet, Avro and CSV.
What, when I worked at Google concatenating protobuf strings was a common way to concatenate protobufs, they are absolutely splittable. People might not know it but there is a reason they are designed like they are, it is to handle big data as you say.
If you mean you can't split a single protobuf, sure, but you don't need to do that, it is like trying to split a single csv line, doesn't mean csv isn't splittable.
Edit: Maybe some of that tooling to work with protobufs are internal only or not a part of the external protobuf packages though.