Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A love letter to the CSV format

github.com

Post reply on HN
Print view View original

151–160 of 711 posts

Re: A love letter to the CSV format

#151

Post by packetlost »

koolba wrote: post #131
packetlost wrote: post #125

Earlier quoted context omitted.

Eh, it really isn't. The format does not lend itself to tabular data, instead the most natural way of representing data involves duplicating the keys N times for each record.

You can easily represent it as an array: [“foo”,”bar”,123] That’s as tabular as CSV but you now have optional types. You can even have lists of lists. Lists of objects. Lists of lists of objects…

You're missing my point: basically nothing spits out data in that format because it's not ergonomic to do so. JSON is designed to represent object hierarchies, not tabular data.

Re: A love letter to the CSV format

#152

Post by mort96 »

bee_rider wrote: post #135
mort96 wrote: post #96

Earlier quoted context omitted.

If have to use a dedicated tabular data editing program, I may as well use a spreadsheet application. What do the options you propose do better than libreoffice calc?

column -t | less -S is pretty nice because you can inspect a file or dataset on a server (no X) before downloading, to see if you even want to bother. Or you can pass it along through a series of pipes to just get the rows you want.

What part of "the value of CSV is to have something that can be easily viewed and modified in any text editor" do you not understand

Re: A love letter to the CSV format

#153

Post by rcbdev »

int_19h wrote: post #134
benwilber0 wrote: post #114

Earlier quoted context omitted.

JSON is a textual encoding no different than CSV. It's just that people tend to use specialized tools for encoding and decoding it instead of like ",".join(row) and row.split(",") I have seen people try to build up JSON strings like that too, and then you have all the same problems. So there is no problem with CSV except that maybe it's too deceptively simple. We also see people trying to build things like URLs and q…

The problem with CSV is that there's no clear standard, so even if you do reach for a library to parse it, that doesn't ensure compatibility.

Same for JSON though. What Python considers a valid JSON might not be that if you ask a Java library.

Re: A love letter to the CSV format

#154

Post by simonw »

koolba wrote: post #131
packetlost wrote: post #125

Earlier quoted context omitted.

Eh, it really isn't. The format does not lend itself to tabular data, instead the most natural way of representing data involves duplicating the keys N times for each record.

You can easily represent it as an array: [“foo”,”bar”,123] That’s as tabular as CSV but you now have optional types. You can even have lists of lists. Lists of objects. Lists of lists of objects…

Right - the JSON-newline equivalent of CSV can look like this:

    ["id", "species", "nickname"]
    [1, "Chicken", "Chunky cheesecakes"]
    [2, "Dog", "Wagging wonders"]
    [3, "Bunny", "Hopping heroes"]
    [4, "Bat", "Soaring shadows"]

Re: A love letter to the CSV format

#155

Post by marcellus23 »

alkh wrote: post #20
nayuki wrote: post #10

I greatly prefer TSV over CSV. https://en.wikipedia.org/wiki/Tab-separated_values

The problem with using TSV is different user configuration. For ex. if I use vim then Tab might indeed be a '\t' character but in TextEdit on Mac it might be something different, so editing the file in different programs can yield different formatting. While ',' is a universal char present on all keyboards and formatted in a single way

The only situation I can think of where a tab is not a tab, is in a code editor that's been configured (possibly by default) to use spaces instead. But that's an easy enough configuration to change. And certainly wouldn't be a problem for something like TextEdit.

Re: A love letter to the CSV format

#156

Post by k_bx »

I've recently been developing a raspberry pi based solution which works with telemetry logs. First implementation used an SQLite database (with WAL log) – only to find it corrupted after just couple of days of extensive power on/off cycles.

I've since started looking at parquet files – which turned out to not be friendly to append-only operations. I've ended up implementing writing events into ipc files which then periodically get "flushed" into the parquet files. It works and it's efficient – but man is it non-trivial to implement properly!

My point here is: for a regular developer – CSV (or jsonl) is still the king.

Re: A love letter to the CSV format

#157

Post by testudovictoria »

mitchpatin wrote: post #39

CSV still quietly powers the majority of the world’s "data plumbing." At any medium+ sized company, you’ll find huge amounts of CSVs being passed around, either stitched into ETL pipelines or sent manually between teams/departments. It’s just so damn adaptable and easy to understand.

Insurance. One of the core pillars of insurance tech is the CSV format. You'll never escape it.

Re: A love letter to the CSV format

#158

Post by dbro »

williamcotton wrote: post #81

Essential CSV shell tools: csvtk: https://bioinf.shenwei.me/csvtk/ gawk: https://www.gnu.org/software/gawk/manual/html_node/Comma-Sep... awk: https://github.com/onetrueawk/awk?tab=readme-ov-file#csv

Forgive me for promoting this that I wrote:

csvquote: https://github.com/dbro/csvquote

Especially for use with existing shell text processing tools, eg. cut, sort, wc, etc.

Re: A love letter to the CSV format

#159

Post by jszymborski »

deathanatos wrote: post #139
jszymborski wrote: post #22

> 4. CSV is streamable This is what keeps me coming back.

…ndjson is streamable, too…

I like ndjson and jsonl just fine, but unless I need a more complicated structure, it's not worth the extra hassle of parsing JSON.

Re: A love letter to the CSV format

#160

Post by 999900000999 »

Xelbair wrote: post #140
999900000999 wrote: post #109

The best part about csv, anyone can write a parser in 30 minutes meaning that I can take data from the early '90s and import it into a modern web service. The worst part about CSV, anyone can ride a parser in about 30 minutes, meaning that it's very easy to get incorrect implementations, incorrect data, and other strange undefined behaviors. But to be clear json, and yaml also have issues with everyone trying to rein…

until you find someone abusing XSD schemas, or someone designing a "dynamically typed" XML... or sneaks in extra data in comments - happened to me way often than it should.

My condolences. Any open standard runs the risk of this happening. It's not a problem I think we'll ever solve.
Post reply on HN