Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A love letter to the CSV format

github.com

Post reply on HN
Print view View original

701–710 of 711 posts

Re: A love letter to the CSV format

#701

Post by nmz »

lyu07282 wrote: post #222
Maro wrote: post #31

I hate CSV (but not as much as XML). Most reasonably large CSV files will have issues parsing on another system.

It makes me a bit worried to read this thread, I would've thought its pretty common knowledge why CSV is horrible and widely agreed upon. I also have hard time taking anybody seriously who uses "specification" and "CSV" in the same sentence unironically. I suspect its 1) people who worked with legacy systems AND LIKED IT, or 2) people who never worked with legacy systems before and need to rediscover painful old less…

The thread makes me worried for a different reason, especially how many avoid reading the RFC and even saying that the RFC doesn't matter, that there's no standard for csv. maybe there should be an extension for rfc-csv.

Re: A love letter to the CSV format

#702

Post by j45 »

thayne wrote: post #669
j45 wrote: post #628

Earlier quoted context omitted.

CSV is a data-exchange format.

But it is terrible at that because there is no widely adhered to standard[1], the sender and receiver often disagree on the details of what exactly a CSV is. [1]: yes, I know about RFC 4180. But csvs in the wild often don't follow it.

Agreed that that CSV construction isn't consistent even with standards.

That footprint seems to be dozens of variations to work with to find a library for?

CSV are universal though, text kind of like markdown, and that is my intended main point.

Re: A love letter to the CSV format

#703

Post by Xelbair »

999900000999 wrote: post #661
marcosdumay wrote: post #568

Earlier quoted context omitted.

In principle, if you make your standard extensible enough, people should stop sneaking data into comments or strings. ... What makes the GP's problem so much more amusing. XML was the last place I'd expect to see it.

That's assuming they know how to use it properly. Rest has this same issue. I've seen this when trying to integrate with 3rd party apis. Status Code 200 Body: Sorry bro, no data. Even then, this is subject to debate. Should a 404 only be used when the endpoint doesn't exist ? When we have no data to return, etc.

i think that default REST is a bit problematic as it conflates transport protocol level errors with application logic ones.

At least unless you use application/problem+json or application/problem+xml MIME types but those are still just in draft stage

https://datatracker.ietf.org/doc/html/rfc9457#name-the-probl...

Re: A love letter to the CSV format

#704

Post by deepsun »

ethbr1 wrote: post #657
deepsun wrote: post #614

Earlier quoted context omitted.

Just a nitpick about consultancy shops -- I've had a chance of working in one in eastern europe and noticed that it's approach to quality was way better than client's. It also helped that client paid by hours, so consultancy company was incentivized to spend more time on refactorings, improvals and testing (with constant pushback from client). So I don't buy the consultancy company sentiment, it always boils down to…

How big was the one you worked for? In my experience, smaller ones tend to align incentives better. Once they grow past a certain size though, it's a labor arbitrage game. Bill client X, staff with resources costing Y (and over-represented), profit = X-Y, minimize Y to maximize profit. PwC / IBM Global Services wasn't offering the best and brightest. (Outside of aforementioned tiger teams)

I agree with you in general, although my case was the other way around. My company was 10k+ people. But my client was probably the most technically advanced company at that time, with famously hard interviews for their own employees. My employer also didn't want to lose the client (it was beginning of collaboration), and since everyone wanted to work there (and move to US+California) my shop applied pretty strong filter for their own heads, even before sending them to client's vendor-interview.

And client was very-very happy with the quality, and that we didn't fight for promotions and could maintain very important, but promotion-poor projects. Up to the point that client trusted to completely gave couple of projects fully to my shop. When you don't need to fight for promotions, code quality also improves.

Re: A love letter to the CSV format

#705

Post by mharig »

mjw_byrne wrote: post #691
recursive wrote: post #643

Earlier quoted context omitted.

"Enterprise software" has been defined as software that is purchased based on the decisions of people that will not use it. I think that explains a lot.

Yep, we had a constant tug of war between techies who wanted to use open-source tools that actually work (Linux, Postgres, Python, Go etc.) and bigwigs who wanted impressive-sounding things in Powerpoint decks and were trying to force "enterprise" platforms like Palantir and IBM BigInsights on us. Any time we were allowed to actually test one of the "enterprise" platforms, we'd break it in a few minutes. And I don't…

[dead]

Re: A love letter to the CSV format

#708

Post by xp84 »

consp wrote: post #427
xp84 wrote: post #385

Earlier quoted context omitted.

I feel like YAML is a spiritual successor to the .ini, since it shares a notable ideal of simple human readability/writability.

YAML is readable? No way as there are too many ways to do the same thing and nested structures are unclear to the non trained eye (what is a list? What is nested?), let alone indentation in large files is an issue especially with the default 2 space unreadable standard so many people adhere to. YAML simple? It's sepc is larger than XML... Parsing of numbers and strings is ambiguous, leading zeros are not strings but…

I agree that one could make wild YAML if you get into advanced stuff, but I make YAML files that look like this:

  things:
    - title: "A thing"
      item_id: "deadbeef-feb1-4e8c-b61c-dd9a7a9fffff"
      is_active: true
      favorite_formats:
        - yml
        - ini
    - title: "Another thing"
      item_id: "deadbeef-feb1-3333-4444-dd9a7a9fffff"
      is_active: false
      favorite_formats:
        - mp3
        - wav
Just because you can use it to create a monstrosity doesn't prevent it from being useful for simple configuration. Basically, it's just prettier JSON.

Re: A love letter to the CSV format

#709

Post by leonim »

saint_yossarian wrote: post #175
williamcotton wrote: post #81

Essential CSV shell tools: csvtk: https://bioinf.shenwei.me/csvtk/ gawk: https://www.gnu.org/software/gawk/manual/html_node/Comma-Sep... awk: https://github.com/onetrueawk/awk?tab=readme-ov-file#csv

Also VisiData is an excellent TUI spreadsheet.

One of my favorite tools. However, I don’t think that Visidata is a spreadsheet, even though it looks like one and is named after one. It is more spreadsheet adjacent. It is focused on row-based and column-based operations. It doesn’t support arbitrary inter-cell operation(s), like you get in Excel-like spreadsheets. It is great for “Tidy Data’, where each row represents a coherent set of information about an object or observation. This is very much like Awk, or other pipeline tools which are also line/row oriented.

For CLI tools, I’m also a big fan of Miller (https://github.com/johnkerl/miller) to filter/modify CSV and other data sources.

Re: A love letter to the CSV format

#710

Post by RandomMarius »

"CSV" should die. The linked article makes critical ommisions and is wrong about some points. Goes to show just how awful "CSV" is.

For one thing, it talks about needing only to quote commas and newlines... qotes are usually fine... until they are on either side of the value. then you NEED to quote them as well.

Then there is the question about what exactly "text" is; with all the complications around Unicode, BOM markers, and LTR/RTL text.

Post reply on HN