Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Friends don't let friends export to CSV

kaveland.no

Post reply on HN
Print view View original

71–80 of 459 posts

Re: Friends don't let friends export to CSV

#71

Post by rgmerk »

fragmede wrote: post #59
barrkel wrote: post #36

Earlier quoted context omitted.

Sure, you tell the finance industry that. They have systems, the systems already produce CSV. They can sign a contract worth multiples of your salary if you can consume it. Do you want the money or not?

In case it wasn't clear, I want software engineers in the finance industry to implement sqlite import and export in their various pieces of software, not too give up on lucrative, existing contracts, obviously.

The inertia around CSV in the finance industry is incredibly strong, and let’s just say some of the biggest industry players aren’t exactly reknowned for their cutting-edge practices when it comes to data interchange formats.

Re: Friends don't let friends export to CSV

#72

Post by chasil »

anymouse123456 wrote: post #38
Closi wrote: post #12

Of course if you only consider the disadvantages, something looks bad. The advantages of CSV are pretty massive though - if you support CSV you support import and export into a massive variety of business tools, and there is probably some form of OOTB support.

This is the biggest win IME. You have a (usually) portable transport format that can get the information into and out of an enormous variety of tools that do not necessarily require a software engineer in the middle. I'm also struggling with such a quick dismissal of human readable formats. It's a huge feature. What happens when there's a problem with a single CSV file in some pipeline that's been happily running fin…

In a POSIX shell, I actually prefer to use the bell character for IFS.

  while IFS="$(printf \\a)" read -r field1 field2...
  do ...
  done
This works just as well as anything outside the range of printing characters.

Getting records that contain newlines would be a bit trickier.

Re: Friends don't let friends export to CSV

#74

Post by nmz »

johnea wrote: post #64

I've written CSV exports in C from scratch, no external dependencies required. It's "Comma Separated Variables", it doesn't really need anymore specification than that. These files have always imported into M$ and libre office suites without issue.

oh boy. here's where it breaks

- supporting "" (single)

- Supporting newlines in "", oops, now you can't getline() and instead need to getdelim()

- Supporting comments # (why is this even a thing)

- Supporting multiple "" in a field

- Escaping " with "" or \"

- length based csv, so all fields are seekable.

It's a mess, which one's your csv?

Re: Friends don't let friends export to CSV

#75

Post by nicbou »

Closi wrote: post #12

Of course if you only consider the disadvantages, something looks bad. The advantages of CSV are pretty massive though - if you support CSV you support import and export into a massive variety of business tools, and there is probably some form of OOTB support.

I also like that Preview can display CSV

Re: Friends don't let friends export to CSV

#76

Post by gregw2 »

barrkel wrote: post #20
rietta wrote: post #5

Or export to CSV correctly and test with Excel and/or LibreOffice. Honestly CSV is a very simple, well defined format, that is decades old and is “obvious”. I’ve had far more trouble with various export to excel functions over the years, that have much more complex third-party dependencies to function. Parsing CSV correctly is not hard, you just can’t use split and be done with it. This has been my coding kata in eve…

CSV is not well-defined. Data in the wild doesn't even agree that it's comma separated. String encoding? Dates? Formatted numbers? Booleans (T/F/Y/N/etc)? Nested quotes? Nested CSV!? How about intermediate systems that muck things up. String encoding going through a pipeline with a misconfiguration in the middle. Data with US dates pasted into UK Excel and converted back into CSV, so that the data is a mix of m/d/yy…

And “do you have a header row”? And null/nil/nul/blank specification/expectation?

Re: Friends don't let friends export to CSV

#77

Post by Cheer2171 »

jjgreen wrote: post #3

An article promoting parquet over CSV. Fair enough, but parquet has been around for a while and still no support in Debian. Is there some deep and dark reason why?

What do you mean there is no parquet support in Debian? Data formats should be supported in userspace and there are plenty of parquet libraries and userspace tools an apt-get away. There is exactly as much support for tar in Debian as there is for parquet.

Re: Friends don't let friends export to CSV

#78

Post by prepend »

Macha wrote: post #51
rietta wrote: post #46

Earlier quoted context omitted.

Localization is a thing. None of this is a show stopper. Subclasses and configuration screen for import and export.

And then you get a business analyst at your client going "I just hit export in our internal tool, what's this delimiter that you're asking me about in the upload form? Google Sheets doesn't ask me to tell them that"

And how do you fix the “some analyst” problem? Is there a better format that reduces this problem?

Re: Friends don't let friends export to CSV

#80

Post by josephg »

prepend wrote: post #73

CSV is very durable. If I want it read in 20 years, csv is the way to go until it’s just too big to matter. Of course there are better formats. But for many use cases friends encourage friends to export to CSV.

Eh. I much prefer to produce and consume line delimited JSON. (Or just raw JSON). Its easy to parse, self descriptive and doesn't have any of CSV's ambiguity around delimiters and escape characters.

Its a little harder to load into a spreadsheet, but in my experience, way easier to reliably parse in any programming language.

Post reply on HN