Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

World's Smallest CSV Parser (C#)

github.com

Post reply on HN
Print view View original

41–50 of 72 posts

Re: World's Smallest CSV Parser (C#)

#41

Post by LeonB »

samatman wrote: post #23

To the author: please consider using an actual license. I infer from the tone of your license that you intended it to give away the code to any "human" who wants to "use" it. What if I modify it? Is modification "use"? (No.) What if a shell script calls it? Is that a "human" using it, or a "computer"? Is linking "use"? The result is probably a license which is non-free, which doesn't appear to be your intention.

Maybe also add that as an issue at the repo.

(Writing it as a comment here is/was still useful because it may help others)

Re: World's Smallest CSV Parser (C#)

#42

Post by nrdvana »

It's a nice tidy CSV parser, but needs a new title. "world's smallest" is never going to happen in C#, for any measure of "smallest". And aside from that, nobody should be rolling their own CSV parsers if they want to solve real-world problems; use the most capable library your language offers you, which will account for a hundred edge cases yours doesn't.

Re: World's Smallest CSV Parser (C#)

#43

Post by xamuel »

userbinator wrote: post #31
xamuel wrote: post #15

Very nice. The submission's main file (SmallestCSVParser.cs) is 3851 characters (of which 657 are commentary). Mine, in C, is only 2807 characters (of which 198 are commentary): https://github.com/semitrivial/csv_parser/blob/master/csv.c Ahh, but the submission's main file is for parsing an entire .csv, whereas mine is only for parsing a single "line" (possibly including quote-escaped newlines). So the submission win…

Do you need more than 1k of additional source to wrap the line parsing in a loop? I doubt it.

Linebreaks can be escaped in CSV, so splitting a file into rows is actually ~1/3 the complexity of parsing a whole row.

See: https://github.com/semitrivial/csv_parser/blob/master/split....

Though I suppose that's the naive approach. You could combine the two into a single file by, like you say, wrapping the row-parser in a (clever, non-trivial) outer loop, and it probably wouldn't take anywhere near 1000 characters to do that...

Re: World's Smallest CSV Parser (C#)

#44

Post by xamuel »

nrdvana wrote: post #42

It's a nice tidy CSV parser, but needs a new title. "world's smallest" is never going to happen in C#, for any measure of "smallest". And aside from that, nobody should be rolling their own CSV parsers if they want to solve real-world problems; use the most capable library your language offers you, which will account for a hundred edge cases yours doesn't.

Funny story re "nobody should be rolling":

When I was switching from academia to industry, I decided, based on HN comments like this, that I should un-publish my CSV parser.

I was worried potential employers would tsk-tsk me for self-rolling.

I promptly got an email from the creator of Ruby asking me why I had un-published my CSV parser, which apparently was being used in Ruby at the time.

(...And then later I landed my current job, a dream job, a large part of which involves handling CSV files in finance!)

Re: World's Smallest CSV Parser (C#)

#45

Post by datascienced »

int_19h wrote: post #34
twoodfin wrote: post #30

Earlier quoted context omitted.

Of course the “hard part” of CSV parsing is dealing with escapes, which break simple splits. But now I’m wondering if a good approach might be to split on the escape character and then reassemble / parse from there, safe in the knowledge that every character has exactly one interpretation.

"Normal" CSV doesn't have escape characters. Quotes in quoted strings are escaped by doubling then, and everything else (including newlines) is interpreted as is inside quoted strings.

There is no normal csv! I always used Excel as the “standard” when writing a CSV parser.

If every field is quoted you can indeed remove the first and last “, then split on “,“ and then replace “” with “ in the fields. Excuse my phone converting the quotes!

Re: World's Smallest CSV Parser (C#)

#46

Post by LeonB »

int_19h wrote: post #34
twoodfin wrote: post #30

Earlier quoted context omitted.

Of course the “hard part” of CSV parsing is dealing with escapes, which break simple splits. But now I’m wondering if a good approach might be to split on the escape character and then reassemble / parse from there, safe in the knowledge that every character has exactly one interpretation.

"Normal" CSV doesn't have escape characters. Quotes in quoted strings are escaped by doubling then, and everything else (including newlines) is interpreted as is inside quoted strings.

There is no spec or standard or consensus on “Normal” CSV.

I like it when CSV follows RFC 4180 too - but it’s descriptive not prescriptive.

Re: World's Smallest CSV Parser (C#)

#47

Post by LeonB »

deathanatos wrote: post #17
lolpanda wrote: post #4

I reviewed the code in this project and it looks pretty reasonable. I thought CSV was a loosely specified format. In my past experience, I never had a smooth experience moving data from one system to another using CSV. I had a lot of trouble with Snowflake -> CSV -> Clickhouse. I now use JSONL for pretty much everything.

There's an RFC that specifies a standard format for CSV. If you're smart you'd use it ^W^W… well, you'd probably not use CSV to start with. The problem is that often, what you have to ingest is more properly described as "malformed CSV / bytes that loosely resembled CSV in some manner that I have no choice but to either try to shove into a parser, or write some custom junk for this hot garbage because it comes form a…

It doesn’t specify a standard format.

It describes a common format.

https://datatracker.ietf.org/doc/html/rfc4180

Re: World's Smallest CSV Parser (C#)

#48

Post by theendisney »

twoodfin wrote: post #30
theendisney wrote: post #18

Earlier quoted context omitted.

Fun! Convert to js: csv.split('\n").join('".split(\',\'));a.push("'); So that each line becomes: a.push("foo,bar,baz".split(',')); And then we have an array of arrays.

Of course the “hard part” of CSV parsing is dealing with escapes, which break simple splits. But now I’m wondering if a good approach might be to split on the escape character and then reassemble / parse from there, safe in the knowledge that every character has exactly one interpretation.

In my sillyness I forgot to brag about the output still being csv.

If you want to enjoy all the strange escapery, extra commas, line breaks and wrapping quotes, you may describe it in code in the first and last 2 columns.

Re: World's Smallest CSV Parser (C#)

#49

Post by int_19h »

datascienced wrote: post #45
int_19h wrote: post #34

Earlier quoted context omitted.

"Normal" CSV doesn't have escape characters. Quotes in quoted strings are escaped by doubling then, and everything else (including newlines) is interpreted as is inside quoted strings.

There is no normal csv! I always used Excel as the “standard” when writing a CSV parser. If every field is quoted you can indeed remove the first and last “, then split on “,“ and then replace “” with “ in the fields. Excuse my phone converting the quotes!

That is precisely why I put "normal" in quotes.

Nevertheless, if there is a way to escape anything at all, usually it is the quotation mark, and usually it is escaped by doubling. Pretty much any other scheme is very unlikely to be properly interpreted in this context.

Re: World's Smallest CSV Parser (C#)

#50

Post by datascienced »

int_19h wrote: post #49
datascienced wrote: post #45

Earlier quoted context omitted.

There is no normal csv! I always used Excel as the “standard” when writing a CSV parser. If every field is quoted you can indeed remove the first and last “, then split on “,“ and then replace “” with “ in the fields. Excuse my phone converting the quotes!

That is precisely why I put "normal" in quotes. Nevertheless, if there is a way to escape anything at all, usually it is the quotation mark, and usually it is escaped by doubling. Pretty much any other scheme is very unlikely to be properly interpreted in this context.

Yes indeed. To make it easy to parse everything has to be quoted. If some things are quoted then you can’t just split on comma because for example:m

    “, is a cat”,”, is my boyfriend”,123
etc.
Post reply on HN