Live data from Hacker News

Sj.h: A tiny little JSON parsing library in ~150 lines of C99

github.com

131–140 of 248 posts

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#131
post #102

Earlier quoted context omitted.

Public facing interfaces are their own special thing, regardless if json or anything else, and not all data is a public facing interface. If you need it, then you need it. But if you don't need it, then you don't need it. There is a non-trivial value in the smallness and simplicity, and a non-trivial cost in trying to handle infinity problems when you don't have infinity use-case.

This is a serialization library. The entire point is to communicate with data that's coming from out of process. It should be safe by default especially if it's adding a quick check to avoid overflow and undefined behavior.

Incorrect assumption.

If you are reading data from a file or stream that only you yourself wrote some other time, then it's true that data could possibly have been corrupted or something, but it's not true that it's automatically worth worrying about enough to justify making the code and thus it's bug surface larger.

How likely is the problem, how bad are the consequences if the problem happens, how many edge cases could possibly exist, how much code does it take to handle them all? None of these are questions you or anyone else can say about anyone else's project ahead of time.

If the full featured parser is too big, then the line drawing the scope of the lightweight parser has to go somewhere, and so of course there will be things on the other side of that line no matter where it is except all the way back at full-featured-parser.

"just this one little check" is not automatially reasonable, because that check isn't automatically more impoprtant than any other, and they are all "just one little checks"s. The one little check would perevent what? Maybe a problem that never happens or doesn't hurt when it does happen. A value might be misinerpreted? So what? Let it. Maybe it makes more sense to handle that in the application code the one place it might matter. If it will matter so much, then maybe the application needs the full fat library.

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#132
post #12

JSON parser libraries in general is a black hole of suffering imo. They're either written with a different use case in mind, or a complex mess of abstractions; often both. It's not a very difficult problem to solve if you only write exactly what you need for your specific use case.

It's astonishing how involved a fucking modern JSON library becomes. The once "very simple" C++ single-header JSON library by nlohmann is now * 13 years old * is still actively merging PRs (last one 5 hours ago) * has 122 __million__ unit tests Despite all this, it's self-admittedly still not the fastest possible way to parse JSON in C++. For that you might want to look into simdjson. Don't start your own JSON parser…

I am very surprised to hear the unit testing statistic. What kind of unholy edge cases would JSON parsing require to make it necessary to cover 122 million variations?

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#133
post #19

The library doesn’t check for signed integer overflow here: https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... Certain inputs can therefore trigger UB.

diff --git a/sj.h b/sj.h index 60bea9e..25f6438 100644 --- a/sj.h +++ b/sj.h @@ -85,6 +85,7 @@ top: return res; case '{': case '[': + if (r->depth > 999) { r->error = "can't go deeper"; goto top; } res.type = (*r->cur == '{') ? SJ_OBJECT : SJ_ARRAY; res.depth = ++r->depth; r->cur++; There, fixed it

I can only hope this was made by an LLM and not a real human.

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#134
post #19

The library doesn’t check for signed integer overflow here: https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... Certain inputs can therefore trigger UB.

Will trigger UB if level depth is > 2 billion or in the 2nd case number of lines > 2 billion.

Limit you JS input to 1 GB. I will have more problems in other portions of the stack if I start to receive a 2 GB JSON file over the web.

And if I still want to make it work for > 2GB, I would change all int in the source to 64 bits. Will still crash if input is > 2^64.

What I won't ever do in my code is check for int overflow.

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#135
post #108

Earlier quoted context omitted.

It's open source, not free software.

‘Free software’ and ‘open source software’ (as respectively defined by the FSF [1] and the OSI [2], which is how they’re usually used in practice) have overlapping definitions. The project in question is released into the public domain via the Unlicense, which qualifies as a free software ‘licence’. Many of the other projects use the MIT/Expat licence, which also qualifies as a free software licence. [1] https://www.…

And how exactly does it not qualify as an open source license? Seems to meet the definition as far as I can see.

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#136

Earlier quoted context omitted.

I mean, what else is there to do when iterating over a JSON file? Delegating number parsing and UNICODE handling to the user can be considered a feature (since I can decide on my own how expensive/robust I want this to be).

Extracting the data into objects. Libraries like Serde and Pydantic do this for you. Hell the original eval() JSON loading method did that too.

Then you lose the ability to do streaming.

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#137
post #19

The library doesn’t check for signed integer overflow here: https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... Certain inputs can therefore trigger UB.

I wouldn't expect a library like this to be secure. If you want it to be memory safe, compile it with Fil-C.

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#138
post #12

JSON parser libraries in general is a black hole of suffering imo. They're either written with a different use case in mind, or a complex mess of abstractions; often both. It's not a very difficult problem to solve if you only write exactly what you need for your specific use case.

It's astonishing how involved a fucking modern JSON library becomes. The once "very simple" C++ single-header JSON library by nlohmann is now * 13 years old * is still actively merging PRs (last one 5 hours ago) * has 122 __million__ unit tests Despite all this, it's self-admittedly still not the fastest possible way to parse JSON in C++. For that you might want to look into simdjson. Don't start your own JSON parser…

This may say more about C++ than JSON

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#139
post #108

Earlier quoted context omitted.

‘Free software’ and ‘open source software’ (as respectively defined by the FSF [1] and the OSI [2], which is how they’re usually used in practice) have overlapping definitions. The project in question is released into the public domain via the Unlicense, which qualifies as a free software ‘licence’. Many of the other projects use the MIT/Expat licence, which also qualifies as a free software licence. [1] https://www.…

And how exactly does it not qualify as an open source license? Seems to meet the definition as far as I can see.

No claim was made that it is not open source. The contention was over if it was a free license or not:

> not free software

which it is. As F3nd0 said, it's both.

Re: Sj.h: A tiny little JSON parsing library in ~150 lines of C99

#140
post #19

The library doesn’t check for signed integer overflow here: https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... https://github.com/rxi/sj.h/blob/eb725e0858877e86932128836c1... Certain inputs can therefore trigger UB.

I wouldn't expect a library like this to be secure. If you want it to be memory safe, compile it with Fil-C.

This has nothing to do with memory safety.
Post reply on HN