Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Serde Rust Framework

serde.rs

Post reply on HN
Print view View original

141–150 of 159 posts

Re: The Serde Rust Framework

#141

Post by the_gipsy »

autarch wrote: post #34
the_gipsy wrote: post #33

Earlier quoted context omitted.

Isn’t it insane to change your types to accommodate this? You have to change all fields accesses, it behaves differently, you lose non-nullability…

> Isn’t it insane to change your types to accommodate this? Welcome to Go! This was a real pain for us at my last job, where we used go-swagger to generate REST API code. All the generated structs had pointers for _non-optional_ fields so it could also check that they were populated. Meanwhile, optional fields were not pointers because they could use the default value. So then we ended up with lots of code doing chec…

Yea exactly, pointers id allowing nils, which I don’t want to conflate with omitting either.

In the end what I did is parse the JSON-string twice, once more with a “checkjson” library that tells me about missing fields.

Re: The Serde Rust Framework

#142

Post by vlovich123 »

pjmlp wrote: post #99
vlovich123 wrote: post #69

Earlier quoted context omitted.

Seems like there’s opportunity for tighter integration with the compiler where serde moves into the compiler or there are acceleration primitives rather than doing the AST macros thing. This sounds similar to the problems of c++ template bloat.

Except C++ template bloat has solutions not yet available in Rust. For example you can pre-instatiate the most common type parameters and just stuck those instantiations into a binary library. Also if you have a nice C++ compiler environment like Visual C++, you can even do edit-and-continue for many kinds of changes, and VS 2022 will double down on that capability.

C++ has theoretical solutions to it. I’ve yet to see anyone actually do anything like that in any professional environment (from small startup codebase a to the largest c++ code bases that exist in the world) because that’s an unmaintainable solution with a high burden placed on the developer.

Whatever the solution Rust comes up with I think will look at the practical component so as to actually see adoption. It seems like the Rust team is very good at studying the mistakes other languages make in their approaches to a problem and finding the right fit within the language/targeted domain.

Re: The Serde Rust Framework

#143

Post by mwcampbell »

brabel wrote: post #134
unanswered wrote: post #40

Earlier quoted context omitted.

Studies have shown that more than 87% of complaints about rust compile time are because people are enabling serde everywhere and for everything.

@burntsushi had a post where he explained that the common error-handling crates also add big compile-time costs. Most projects use them as well as serde, so the problem compounds.

Can you provide a link to that post?

Re: The Serde Rust Framework

#144

Post by spoiler »

brabel wrote: post #135
spoiler wrote: post #115

Earlier quoted context omitted.

How is it implemented in Java or C# if not via runtime reflection? I've always assumed that's the way its done...

Java has annotation processors which can perform code generation, just like Serde does. Manifold [1] an example framework that relies heavily on that, achieving similar things like Serde. Java developers don't like to use these things too much though, unlike Rust developers who love their `#derive`, so you're mostly right that reflection-based serialization is still more common, but that's by choice. Dependency Injec…

Thank you for the detailed write up! I love finding an out about cool language feature/tools

Re: The Serde Rust Framework

#145

Post by spoiler »

Const-me wrote: post #136
spoiler wrote: post #115

Earlier quoted context omitted.

How is it implemented in Java or C# if not via runtime reflection? I've always assumed that's the way its done...

I don't program Java, but I have implemented and used similar things for C#. Reflection is indeed used there, but not to serialize. It's only used once per type, to generate code. C# has multiple ways to generate code in runtime. One is Reflection.Emit, allows to manually generate bytecode instructions + metadata. For instance, one can build new types in runtime. A typical pattern is implementing manually-written abs…

Ah the bytecode generation via Reflection.Emit seems like a really powerful feature. Does it feel "brittle" to use, or is it expressive enough to aid writing more robust code (to a degree, I realise it's not possible to aliminate every problem)?

There's a weird (not in a bad way) elegance here that reminds me of lisp.

Re: The Serde Rust Framework

#146

Post by jlrubin »

one thing that has kinda irked me about serde is that there's relatively limited support for bounding memory usage for types like a Vec at deserialize time without a lot of manual work.

It is doable (e.g. see SafePSBT deserializer here https://github.com/sapio-lang/sapio/blob/master/ctv_emulator...), just not particularly ergonomic.

Re: The Serde Rust Framework

#147

Post by jlrubin »

jlrubin wrote: post #146

one thing that has kinda irked me about serde is that there's relatively limited support for bounding memory usage for types like a Vec at deserialize time without a lot of manual work. It is doable (e.g. see SafePSBT deserializer here https://github.com/sapio-lang/sapio/blob/master/ctv_emulator... ), just not particularly ergonomic.

addendum: if you're like 'when could you want this?', imagine you are deserializing from a Reader which has an internal circular buffer or something from the network (bounded) and you want to limit a peers total resident memory (or something) on your server. Very difficult to do this with Serde!

Re: The Serde Rust Framework

#148

Post by brabel »

mwcampbell wrote: post #143
brabel wrote: post #134

Earlier quoted context omitted.

@burntsushi had a post where he explained that the common error-handling crates also add big compile-time costs. Most projects use them as well as serde, so the problem compounds.

Can you provide a link to that post?

https://www.reddit.com/r/rust/comments/gj8inf/rust_structuri...

Re: The Serde Rust Framework

#149

Post by pjmlp »

vlovich123 wrote: post #142
pjmlp wrote: post #99

Earlier quoted context omitted.

Except C++ template bloat has solutions not yet available in Rust. For example you can pre-instatiate the most common type parameters and just stuck those instantiations into a binary library. Also if you have a nice C++ compiler environment like Visual C++, you can even do edit-and-continue for many kinds of changes, and VS 2022 will double down on that capability.

C++ has theoretical solutions to it. I’ve yet to see anyone actually do anything like that in any professional environment (from small startup codebase a to the largest c++ code bases that exist in the world) because that’s an unmaintainable solution with a high burden placed on the developer. Whatever the solution Rust comes up with I think will look at the practical component so as to actually see adoption. It seem…

We used to do such stuff at Nokia, on HP-UX with Clearcase and .o build caching supported by cleartool using aCC, while working on NetAct, professional enough?

Re: The Serde Rust Framework

#150

Post by JoelMcCracken »

nicopappl wrote: post #123

Two things make serde great: (1) It's format agnostic (2) everyone in the community uses it. My experience with other language ecosystem (I'm thinking Haskell or Scala) is that serialization/deserialization libraries are limited to one format, with incompatible APIs. Even if the libraries in those ecosystem provide the same kind of guarantees, it still doesn't compare to serde. Typically any framework or library (cat…

Ah, cool. If you implement ToJSON from aeson for a type in Haskell, you also can do yaml serialization without any extra work (besides bringing in the YAML library, but the point is that the yaml library reuses ToJSON), but I don't know about alllll of those others that I see here: https://serde.rs/#data-formats)

Pretty cool!

Post reply on HN