Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Friends don't let friends export to CSV

kaveland.no

Post reply on HN
Print view View original

331–340 of 459 posts

Re: Friends don't let friends export to CSV

#331

Post by vouaobrasil »

prepend wrote: post #167
chaps wrote: post #141

Earlier quoted context omitted.

Why is this getting downvoted? They're right that the criticism is pretty excessive: "Maybe his next article will be about how airlines should speak Esperanto because English is such a flawed language. That’s a clever and unique observation." Hm.

I got a little snarky but I think the analogy holds. Esperanto is a superior language to English. And English has many flaws. Theoretically it would be better to have all pilots and airports learn an efficient language. But it would be stupid and immature to seriously write a blog post about that, especially without talking about all the flaws in that plan.

> Esperanto is a superior language to English. And English has many flaws.

At least one one perspective, English is superior. That perspective is that you can actually use it in almost any modern situation because it has been tried and tested globally.

Re: Friends don't let friends export to CSV

#332

Post by alexvoda »

doctor_eval wrote: post #168
lostlogin wrote: post #148

Earlier quoted context omitted.

> Are there unicode characters specifically for delimiters? We could use the HL7 pipe ‘|’ and all enjoy that hell.

It’s been years since I last worked with HL7. Isn’t there also ^ and ~ to deal with? Hell indeed.

Isn't that what HL7 stands for? Hell Layer 7 as in the seventh circle of hell.

Re: Friends don't let friends export to CSV

#333

Post by pif »

diogocp wrote: post #129
hn_throwaway_99 wrote: post #111

Earlier quoted context omitted.

Totally agree. His arguments are basically "performance!" (which is honestly not important to 99% of CSV export users) and "It's underderspecified!" And while I can agree with the second, at least partly, in the real world the spec is essentially "Can you import it to Excel?". I'm amazed at how much programmers can discount "It already works pretty much everywhere" for the sake of more esoteric improvements. All that…

> in the real world the spec is essentially "Can you import it to Excel?" And the answer to that is always no. You will it think it's yes because it works for you, but when you send it to someone who has a different Excel version or simply different regional settings, it won't work. The recipient will first have to figure out what dialect you used to export.

>> in the real world the spec is essentially "Can you import it to Excel?"

> And the answer to that is always no.

Sorry, you are wrong! You are confusing "No" and "Yes, after a quick setup at most".

Re: Friends don't let friends export to CSV

#334

Post by braiamp »

pif wrote: post #333
diogocp wrote: post #129

Earlier quoted context omitted.

> in the real world the spec is essentially "Can you import it to Excel?" And the answer to that is always no. You will it think it's yes because it works for you, but when you send it to someone who has a different Excel version or simply different regional settings, it won't work. The recipient will first have to figure out what dialect you used to export.

>> in the real world the spec is essentially "Can you import it to Excel?" > And the answer to that is always no. Sorry, you are wrong! You are confusing "No" and "Yes, after a quick setup at most".

Oh oh oh, I have a story about a quick setup. Sent a csv file to someone in the same org. Guy said that it was not opening. Wanted me to come to their office to see. I told them that IT support should fix it, since I can't, and every machine on my OU could read it. I was the bad guy. Yeah, quick setup my ass. Users can't be arsed to understand that their setup isn't properly configured.

Re: Friends don't let friends export to CSV

#336

Post by adammarples »

vaylian wrote: post #243
josefx wrote: post #235

Earlier quoted context omitted.

Does it have nesting operators? I want to embedd ASCII within my ASCII fields. So I can have a table within my table.

It kind of does. See `man ascii` * FS (0x1C) file separator * GS (0x1D) group separator * RS (0x1E) record separator * US (0x1F) unit separator I've never seen these in the wild though.

Not seeing them in the wild is good, it means they will work when you use them. The more they get used the more often you'll find they crop up in the text fields you're trying to separate and the whole csv escaping nightmare will continue.

Re: Friends don't let friends export to CSV

#337

Post by braiamp »

Openrefine has saved my bacon more times that I care to admit. It ingest everything and have powerful exporting tools. Friends give friends CSV files, and also tell them about tools that help them deal with wide array of crap formats.

Re: Friends don't let friends export to CSV

#338

Post by adammarples »

Dalewyn wrote: post #211
deathanatos wrote: post #181

Earlier quoted context omitted.

> Special Unicode Characters(tm) can't be typed by anyone on any keyboard and readable by noone. While I'm not a fan of USV, I do believe it is type-able on both macOS and Linux¹. The IME's character picker in both OSes contains all of the necessary characters, I think. (I use "␤" in commit messages, occasionally. That was a typed ␤, not copy/pasted from a Google query or such.) It would be quite tedious , I do admit…

Any character within reason can certainly be entered by way of Character Map in Windows or its equivalent in Linux or MacOS, but if you're arguing that then you don't understand the crux of my argument: Convenience is a virtue. There is value in the delimiter simply being a key on practically any keyboard in existence. Anything that involves something more complicated than just pushing a single button on a keyboard (…

The fact that it is a character on the keyboard is exactly the problem, too. Any character a user can easily enter will definitely end up mixed into your data somewhere.

Re: Friends don't let friends export to CSV

#339

Post by wouldbecouldbe »

Freak_NL wrote: post #279
diogocp wrote: post #129

Earlier quoted context omitted.

> in the real world the spec is essentially "Can you import it to Excel?" And the answer to that is always no. You will it think it's yes because it works for you, but when you send it to someone who has a different Excel version or simply different regional settings, it won't work. The recipient will first have to figure out what dialect you used to export.

Oh absolutely. Don't forget about the giant mess Microsoft made in countries like the Netherlands, where Excel will refuse to automatically open comma-separated values files (you know, CSV), unless the separator is a semicolon — because someone in the past thought that was how the Dutch did it. You want people to be able to open your nice and sensible CSV files in Excel? First figure out which arcane locale specific…

US users have the same in my experience. I've had multiple clients complain the exports are broken, because Excel only imports them (perfectly) but doesn't allow to open them. I thought it was their way of forcing excel files. Iphone & Google Drive no issues.

Re: Friends don't let friends export to CSV

#340

Post by xeromal »

hn_throwaway_99 wrote: post #111
prepend wrote: post #85

This article seems written by someone who never had to work with diverse data pipelines. I work with large volumes of data from many different sources. I’m lucky to get them to send csv. Of course there are better formats, but all these sources aren’t able to agree on some successful format. Csv that’s zipped is producible and readable by everyone. And that makes is more efficient. I’ve been reading these “everyone i…

Totally agree. His arguments are basically "performance!" (which is honestly not important to 99% of CSV export users) and "It's underderspecified!" And while I can agree with the second, at least partly, in the real world the spec is essentially "Can you import it to Excel?". I'm amazed at how much programmers can discount "It already works pretty much everywhere" for the sake of more esoteric improvements. All that…

Check out Polars in python if you want some CSV performance lol. I recently got a 5 million row CSV from a 3rd party and I could manipulate columns (filtering, sorting, grouping) in actions that took less than a second. It's an incredible tool.
Post reply on HN