Live data from Hacker News

Compile-time JSON deserialization in C++

medium.com

51–60 of 89 posts

Re: Compile-time JSON deserialization in C++

#51
post #6

I use this static reflection hack in C++ -- https://godbolt.org/z/enh8za4ja You do have to tag struct fields with a macro, but you can attach contexpr-visitable attributes. There's also a static limit to how many reflectable fields you can have, all reflectable fields need to be at the front of the struct, and the struct needs to be an aggregate.

Holy crap; that's pretty epic. Did you come up with that yourself?

Re: Compile-time JSON deserialization in C++

#52

I think the value of compile-time JSON deserialization is... well I was going to say zero but really it's negative. It's a cute trick, but please don't ever do this in a real project.

So I have owned a library for 6years or so that does constexpr JSON to data structures, JSON Link. There are a few benefits and in the near future with #embed it gets even better. The big benefit is that we can now get earlier errors and do testing in constexpr that gives more guarantees around the areas of core UB and in most implementations they add constexpr checked preconditions on the std library too. But, just because it is marked constexpr, doesn't mean it will be run at compile time. This also, limits the shenanigans that the library dev can do to get potential perf and work around design limitations.

In JSON Link's case, since it was using C++17 at the time, it forced me to think around the problem of allocation and who does it. The library does not allocate, but potentially the data structures being deserialized to will. In C++20 you can get limited constexpr allocations but they are good for things like stacks and eliminating the fixed buffers many devs have used in the past; which is a good thing on it's own but isn't really allowing one to parse to a vector at compile time(as in OP's example) for things that persist.

Where this will get really interesting, though, is when #embed is in the major compilers. It's mostly there in clang, with gcc on the way I believe. It will open the door for DSL's and compile time configs in human readable formats or interop with other tools(maybe GUI designers)

As for OP's library, I am not a fan of the json_value like library approach that treats JSON as a thing to care about when it is usually just an imp detail to move to ones business objects.

TL;DR The big benefit though, is the ability to reason about the quality of the code in the library and have stronger testing.

Re: Compile-time JSON deserialization in C++

#53

I think the value of compile-time JSON deserialization is... well I was going to say zero but really it's negative. It's a cute trick, but please don't ever do this in a real project.

So I have owned a library for 6years or so that does constexpr JSON to data structures, JSON Link. There are a few benefits and in the near future with #embed it gets even better. The big benefit is that we can now get earlier errors and do testing in constexpr that gives more guarantees around the areas of core UB and in most implementations they add constexpr checked preconditions on the std library too. But, just…

You're right to point out that this is really 'first class JSON', rather than the Pydantic/Jackson type thing where the json barely exists and is immediately transformed into your models and classes.

Thanks for reading the article though, that's cool. I am a daw_json_link fan

Re: Compile-time JSON deserialization in C++

#54
post #53

Earlier quoted context omitted.

So I have owned a library for 6years or so that does constexpr JSON to data structures, JSON Link. There are a few benefits and in the near future with #embed it gets even better. The big benefit is that we can now get earlier errors and do testing in constexpr that gives more guarantees around the areas of core UB and in most implementations they add constexpr checked preconditions on the std library too. But, just…

You're right to point out that this is really 'first class JSON', rather than the Pydantic/Jackson type thing where the json barely exists and is immediately transformed into your models and classes. Thanks for reading the article though, that's cool. I am a daw_json_link fan

Build up the test suite inside static asserts, or a macro that lets you switch. It will be really nice when you update right and your IDE will tell you before you even hit compile because clangd found an issue.

Re: Compile-time JSON deserialization in C++

#55
post #22

Not a C++ user, but is this the same as #. reader macro in Common Lisp?

Yes, the #. reader macro is one of the ways how you can achieve this in Common Lisp. Using the reader macro is also way more efficient because you don't awkwardly use your compiler as an interpreter for a weird subset of your actual language - you simply call to compiled code. Seeing Greenspun's tenth rule [1] in action again and again is one of the weird things we Common Lisp programmers have to endure. I wish we wo…

>I wish we would have more discussions on how to improve Lisp even further instead of trying to 'fix' C or C++ for the umpteenth time.

I agree one million percent; projects like SBCL are great, but my impression is that there are tons of improvements to be had in producing optimized code for modern processors (cache friendliness, SIMD, etc), GPU programming etc. I asked about efforts in those directions here and there, but did not get very clear answers.

Re: Compile-time JSON deserialization in C++

#56

I think the value of compile-time JSON deserialization is... well I was going to say zero but really it's negative. It's a cute trick, but please don't ever do this in a real project.

So I have owned a library for 6years or so that does constexpr JSON to data structures, JSON Link. There are a few benefits and in the near future with #embed it gets even better. The big benefit is that we can now get earlier errors and do testing in constexpr that gives more guarantees around the areas of core UB and in most implementations they add constexpr checked preconditions on the std library too. But, just…

Since you've done this for real in a library, I have to ask: how would you decide to use a compile-time template solution like this versus a code generator or some other "outboard" tool to generate code?

I'm curious since I've gone back and forth on this in my own career. Both approaches come with their own pros and cons, but each get us to the same place.

Re: Compile-time JSON deserialization in C++

#57
What a beautiful example of abuse of C++ templates. I love it.

But please don’t do this in production.

What ever you need to do, use C++ templates as the last resort because you’ve figured out all other approaches suck even more. Maintaining template heavy code is absolutely horrible and wasteful (and if it’s C++ production code we measure it’s lifetime in decades). And no, there is no way ”to do it correctly so it doesn’t suck”.

Templates belong to the lowest abstraction levels - as stl mostly does. Anyhting more prevalent is an abomination.

If the schema is fixed, have types with the data and if you have a default data, provide it using initializer lists.

Ie. have a struct or structs with explicit serializeToJson and deserializeFromJson functions.

It’s faster to write than figuring out the correct template gymnastics and about 100x easier to maintain and extend.

Re: Compile-time JSON deserialization in C++

#58
post #13
post #5

Could this be leveraged to emit a parser that is specialized for the provided type that can be used at runtime? Afaik .NET does something like that using code generators. The advantage being that the parser is tailored to the specific type that is deserialized and it writes directly to the struct's fields instead of going through some dictionary.

This lib does something like that: https://github.com/beached/daw_json_link

It seems that you have to maintain hand-coded mappings for each type. Maybe this could be solved by using C++23's compile-time reflections.

Re: Compile-time JSON deserialization in C++

#59
post #34

Earlier quoted context omitted.

Is this real? It can't be real. Nobody can be this stupid. But then again it takes a special kind of person who doesn't understand satire to actually do something like that. Somebody, where they would say "we trained him wrong on purpose as a kind of a joke".

Nah I've seen this happen IRL. In this system "configuration" was read out of tables in a word document, processed via XSLT transformations and eventually it would spit out a huuuuge single C# document (recent "improvement", before that it was some obscure licenced language). Builds happened overnight because they took so long, and there was no way to test something locally. The "advantage" of this system was that th…

> In this system "configuration" was read out of tables in a word document, processed via XSLT transformations and eventually it would spit out a huuuuge single C# document

This is hilarious! It takes a special kind of ignorance to come up with a solution like this.

Re: Compile-time JSON deserialization in C++

#60
post #28

Earlier quoted context omitted.

I recently actually tried to do a very similar thing, although a bit tighter in scope. What stopped me what that actually deserializing floating points cannot currently be done at compile time; the only utility available to do so is `from_chars` and it is only constexpr for ints. I did not see any mention of this in the post; so are you actually simply extracting the string versions of the numbers, without verifying…

I was able to do the primitive long double result = 0.0; while (...) { if (json[head] == '.') ... result *= 10; result += json[head] - '0'; } in a constexpr function with no problem :)

The problem with this is that it will not actually parse double in IEEE 754, as you will accumulate inaccuracies at every step of the loop. In theory, when parsing a float, you are supposed to return the floating point that is closest to the original string number. Your code will not do that. Even if you accept the inaccuracy, if you for some reason load the JSON using a runtime library, you'll get different numbers and consequently and result that depend on those numbers. For my use-case this was not acceptable unfortunately..
Post reply on HN