Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Compile-time JSON deserialization in C++

medium.com

Post reply on HN
Print view View original

31–40 of 89 posts

Re: Compile-time JSON deserialization in C++

#31

Post by dctwin »

worstspotgain wrote: post #7
dctwin wrote: post #2

Hello! I wrote this short blog post about using pattern-matching-like template metaprogramming to deserialize JSON at build time - please let me know what you think (especially if you see improvements)

As someone who used to have to do this sort of compile-time stuff with previous versions of the standard, I'm jealous of how much more can be done now that I don't have to. If you're looking for an interesting follow-up project, here's something I had to do once that's now become much easier: compute a compile-time hash of the compilation for the current translation unit, e.g. __BASE_FILE__ hashed together with __TIM…

Thanks for the idea! Yes constexpr std::vector feels like cheating

Re: Compile-time JSON deserialization in C++

#33

Post by chipdart »

stephc_int13 wrote: post #9

I am afraid of the compile-time cost. For this kind of things I tend to prefer using a simpler program (written in anything you like) to generate C or C++ instead of having the compile do the same thing much slowly. Meta programming can be good, but it is even better done with an actual meta program, IMO.

> I am afraid of the compile-time cost.

Even though compilation time is the bane of C++, I think this concern regarding this specific usage is grossly overblown. I'm going to tell you why.

With incremental builds you only rebuild whatever has changed in your project. Embedding JSON documents in a C++ app is the kind of thing that is rarely touched, specially if all your requirements are met by serializing docs at compile time. This means that this deserialization will only be rarely rebuilt, and only under two scenarios: full rebuild, and touching the file.

As far as full rebuilds go, there is no scenario where deserializing JSON represents a relevant task in your build tree.

As for touching the file, if for some weird and unbelievable reason the build step for the JSON deserialization component is deemed too computationally expensive, it's trivial to move this specific component into a subproject that's built independently. This means that the full cost of an incremental build boils down to a) rebuilding your tiny JSON deserialization subproject, b) linking. Step a) runs happily in parallel with any other build task, thus it's impact is meaningless.

To read more on the topic, google for "horizontal architecture", a concept popularized by the book "Large-Scale C++: Process and Architecture, Volume 1" By John Lakos.

Mountain out of a molehill.

Re: Compile-time JSON deserialization in C++

#34

Post by cdirkx »

threatripper wrote: post #11
adolph wrote: post #10

Earlier quoted context omitted.

Brings to mind the old story about a JSON DSL https://thedailywtf.com/articles/the-inner-json-effect

Is this real? It can't be real. Nobody can be this stupid. But then again it takes a special kind of person who doesn't understand satire to actually do something like that. Somebody, where they would say "we trained him wrong on purpose as a kind of a joke".

Nah I've seen this happen IRL. In this system "configuration" was read out of tables in a word document, processed via XSLT transformations and eventually it would spit out a huuuuge single C# document (recent "improvement", before that it was some obscure licenced language). Builds happened overnight because they took so long, and there was no way to test something locally.

The "advantage" of this system was that there was no need for programmers, as there was "no code", just configuration!. This was supposed to allowed "domain experts" without programming knowledge to work with the system. However a month long training by the creator of the system was still required, as he had to explain which of the 7 boolean types you should use if you wanted to add a new column 0.o (for those who want to know, there was true/false, 0/1, yes/no, true/false/unknown, true/false rendered as a toggle, true/false rendered as a checkbox...)

Re: Compile-time JSON deserialization in C++

#35

Post by kookybakker »

dctwin wrote: post #28
svalorzen wrote: post #21

Earlier quoted context omitted.

I recently actually tried to do a very similar thing, although a bit tighter in scope. What stopped me what that actually deserializing floating points cannot currently be done at compile time; the only utility available to do so is `from_chars` and it is only constexpr for ints. I did not see any mention of this in the post; so are you actually simply extracting the string versions of the numbers, without verifying…

I was able to do the primitive long double result = 0.0; while (...) { if (json[head] == '.') ... result *= 10; result += json[head] - '0'; } in a constexpr function with no problem :)

How about floats in scientific notation?

Re: Compile-time JSON deserialization in C++

#36

Post by Joker_vD »

_nalply wrote: post #19
threatripper wrote: post #11

Earlier quoted context omitted.

Is this real? It can't be real. Nobody can be this stupid. But then again it takes a special kind of person who doesn't understand satire to actually do something like that. Somebody, where they would say "we trained him wrong on purpose as a kind of a joke".

It's the inner platform effect. When I was young I fell into the same trap. I invented a flexible database schema where I put each field into a database row with some metadata describing the field. But that's nonsense. Just use what the database provides. There's a Wikipedia page about it: https://en.wikipedia.org/wiki/Inner-platform_effect A variant of it is: Any sufficiently complicated program contains a slow and…

> flexible database schema where I put each field into a database row with some metadata describing the field.

I imagine everyone has invented this scheme at one point or another. It's so obvious, when you think about it!

Re: Compile-time JSON deserialization in C++

#37

Post by Joker_vD »

kazinator wrote: post #17
stephc_int13 wrote: post #9

I am afraid of the compile-time cost. For this kind of things I tend to prefer using a simpler program (written in anything you like) to generate C or C++ instead of having the compile do the same thing much slowly. Meta programming can be good, but it is even better done with an actual meta program, IMO.

> generate C or C++ instead of having the compile do the same thing much slowly That's a wild-assed guess. A JSON decoder right in the compiler could easily be faster than generation involving extra tool invocations and multiple passes. Also, if you use ten code generators for ten different features in a pipeline instead of ten compile-time things built into the language, will that still be faster? What if most files…

> A JSON decoder right in the compiler could easily be faster than generation involving extra tool invocations and multiple passes.

It also can easily be slower: C++ templates are not exactly known for their blazingly fast compilation speed. Besides, the program they encode in this case is effectively being interpreted by the C++ compiler which, I suppose, is not really optimized for that: it's still mostly oriented around emitting optimized machine code.

Re: Compile-time JSON deserialization in C++

#38

Post by silon42 »

dctwin wrote: post #27
stephc_int13 wrote: post #9

I am afraid of the compile-time cost. For this kind of things I tend to prefer using a simpler program (written in anything you like) to generate C or C++ instead of having the compile do the same thing much slowly. Meta programming can be good, but it is even better done with an actual meta program, IMO.

Yes, I agree. I don't see much practical use in this. I was just surprised how (relatively) straightforwards this is to do, and thought it was more cool than useful

Often I also find the opposite problem ... sure, you can do some stuff in (c++) metaprogramming, but can you (at compile time) generate a JSON/XML/YAML file that can be fed to some other part of the system?

Re: Compile-time JSON deserialization in C++

#40

Post by goodcanadian »

emmanueloga_ wrote: post #16
dctwin wrote: post #2

Hello! I wrote this short blog post about using pattern-matching-like template metaprogramming to deserialize JSON at build time - please let me know what you think (especially if you see improvements)

The concept reminds me of F#'s "Type Providers" [1]. In terms of the implementation ... I feel like C++ is best when used in an "orthodox style" and minimizing the use of templates as much as possible. -- 1: https://learn.microsoft.com/en-us/dotnet/fsharp/tutorials/ty...

My experience with template meta-programming: it is hardly ever useful, but on those rare occasions when it is, it is magical!
Post reply on HN