Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

World's Smallest CSV Parser (C#)

github.com

Post reply on HN
Print view View original

11–20 of 72 posts

Re: World's Smallest CSV Parser (C#)

#11

Post by Jerrrry »

giaour wrote: post #9
neonsunset wrote: post #8

Earlier quoted context omitted.

C# `char` is a UTF-16 code unit. It does not indicate a byte which is just `byte`. Having StringBuilder be a private field on the parser instance is not an issue either - it is simply reused.

Iterating over the `char`s does not support the full range of what can be stored in a C# string (for instance, UTF-8 graphemes that are serialized as surrogate pairs are usually two `char`s in a C# string. .Net provides a TextElementEnumerator that will iterate over graphemes instead: https://learn.microsoft.com/en-us/dotnet/api/system.globaliz... There's a fairly comprehensive guide to working with .net character en…

You imply that a string, reversed, would have the same length as the original.

This is not true.

Re: World's Smallest CSV Parser (C#)

#12

Post by esdf »

Jerrrry wrote: post #11
giaour wrote: post #9

Earlier quoted context omitted.

Iterating over the `char`s does not support the full range of what can be stored in a C# string (for instance, UTF-8 graphemes that are serialized as surrogate pairs are usually two `char`s in a C# string. .Net provides a TextElementEnumerator that will iterate over graphemes instead: https://learn.microsoft.com/en-us/dotnet/api/system.globaliz... There's a fairly comprehensive guide to working with .net character en…

You imply that a string, reversed, would have the same length as the original. This is not true.

Where are they implying this and why would the strings not have the same length? Is there normalization implied somewhere?

Re: World's Smallest CSV Parser (C#)

#13

Post by taspeotis »

DanielBryars wrote: post #3

What's the utility of defining the "Error" exception. Why not use an existing one, say InvalidOperationException, or a plain Exception. Is making your own better practice?

> Why not use ... a plain Exception.

It is forbidden.

https://learn.microsoft.com/en-us/dotnet/standard/exceptions...

> Exception ... None (use a derived class of this exception).

https://learn.microsoft.com/en-us/dotnet/standard/design-gui...

> DO NOT throw System.Exception or System.SystemException.

Re: World's Smallest CSV Parser (C#)

#14

Post by mbreese »

lolpanda wrote: post #4

I reviewed the code in this project and it looks pretty reasonable. I thought CSV was a loosely specified format. In my past experience, I never had a smooth experience moving data from one system to another using CSV. I had a lot of trouble with Snowflake -> CSV -> Clickhouse. I now use JSONL for pretty much everything.

This is the problem… CSV isn’t specified at sufficient detail, it is just too loose in the real world. So the question “can you make a small parser” isn’t a real issue. And then, the problem with such a small parser is — which edge cases are you missing/ignoring?

I just don’t see the flex in having a small csv parser.

Re: World's Smallest CSV Parser (C#)

#15

Post by xamuel »

Very nice. The submission's main file (SmallestCSVParser.cs) is 3851 characters (of which 657 are commentary).

Mine, in C, is only 2807 characters (of which 198 are commentary):

https://github.com/semitrivial/csv_parser/blob/master/csv.c

Ahh, but the submission's main file is for parsing an entire .csv, whereas mine is only for parsing a single "line" (possibly including quote-escaped newlines). So the submission wins :)

Re: World's Smallest CSV Parser (C#)

#16

Post by neilv »

lolpanda wrote: post #4

I reviewed the code in this project and it looks pretty reasonable. I thought CSV was a loosely specified format. In my past experience, I never had a smooth experience moving data from one system to another using CSV. I had a lot of trouble with Snowflake -> CSV -> Clickhouse. I now use JSONL for pretty much everything.

Yes, loosely specified in practice. I made a CSV parser that tries to do something reasonable for many variants by default. When that's not enough, you can specify options. https://www.neilvandyke.org/racket/csv-reading/

Re: World's Smallest CSV Parser (C#)

#17

Post by deathanatos »

lolpanda wrote: post #4

I reviewed the code in this project and it looks pretty reasonable. I thought CSV was a loosely specified format. In my past experience, I never had a smooth experience moving data from one system to another using CSV. I had a lot of trouble with Snowflake -> CSV -> Clickhouse. I now use JSONL for pretty much everything.

There's an RFC that specifies a standard format for CSV. If you're smart you'd use it ^W^W… well, you'd probably not use CSV to start with.

The problem is that often, what you have to ingest is more properly described as "malformed CSV / bytes that loosely resembled CSV in some manner that I have no choice but to either try to shove into a parser, or write some custom junk for this hot garbage because it comes form a source that I cannot control".

A lot of parsers are fairly configurable precisely to account for the situation of "the other end is sending me ill-defined jank" and to be flexible enough that maybe, just maybe, it'll mostly work. But it's hardly "engineering" at that point.

Re: World's Smallest CSV Parser (C#)

#18

Post by theendisney »

cb321 wrote: post #2

Don't just parse - convert. In a pipeline to split-parseable data, if you like, such as the possibly smaller, faster, and more general: https://github.com/c-blake/nio/blob/main/utils/c2tsv.nim (And, ideally, convert all the way to a mmap & go binary format like nio so you don't have to re-parse.)

Fun! Convert to js:

csv.split('\n").join('".split(\',\'));a.push("');

So that each line becomes:

a.push("foo,bar,baz".split(','));

And then we have an array of arrays.

Re: World's Smallest CSV Parser (C#)

#19

Post by recursive »

taspeotis wrote: post #13
DanielBryars wrote: post #3

What's the utility of defining the "Error" exception. Why not use an existing one, say InvalidOperationException, or a plain Exception. Is making your own better practice?

> Why not use ... a plain Exception. It is forbidden. https://learn.microsoft.com/en-us/dotnet/standard/exceptions... > Exception ... None (use a derived class of this exception). https://learn.microsoft.com/en-us/dotnet/standard/design-gui... > DO NOT throw System.Exception or System.SystemException.

I'm not seeing where it says not to throw InvalidOperationException.

Re: World's Smallest CSV Parser (C#)

#20

Post by Jerrrry »

esdf wrote: post #12
Jerrrry wrote: post #11

Earlier quoted context omitted.

You imply that a string, reversed, would have the same length as the original. This is not true.

Where are they implying this and why would the strings not have the same length? Is there normalization implied somewhere?

If they weren't reversing it, what other operation would separate grapheme clusters?
Post reply on HN