Live data from Hacker News

The Serde Rust Framework

serde.rs

131–140 of 159 posts

Re: The Serde Rust Framework

#131
I know it's nit picking but I would say it's a library not a framework.

There are varying definitions of framework but what tend to be common is that they have a strong effect of "locking you into their ecosystem", which isn't really the case for serde.

Re: The Serde Rust Framework

#132
post #118

Earlier quoted context omitted.

The only downside is compile time bloat. Serde generates heaps and heaps of generic code. This all gets optimized away to be very efficient, but only once it reaches LLVM. Ever tried working on a crate with hundreds or thousands of de/serializable types? Compile times shoot through the roof really quickly, and serde is often the culprit. The maintainer of serde also created `miniserde` [1] to tackle this problem, whi…

It's a tradeoff. Switch to Python and watch your serialization code become at least 10x slower at runtime.

The problem is IMHO less the increased runtime cost, but that it also tends to get less reliable, robust, harder to get actual right (instead of just somehow working) etc.

Re: The Serde Rust Framework

#133
post #100

Earlier quoted context omitted.

Nah, not like it is in Rust. For the vast majority of projects, serde makes parsing / serializing something they don't even have to think about , without having to sacrifice type safety to get there. Until you experience that in a large project with a lot of moving parts and developers, it's hard to appreciate just how many developer cycles you were wasting before on stuff that can be totally automated. It's format-a…

That has been the same in Java and .NET world for 20 years now, just plug the desired serializer from the standard library and be done with it.

Yay, except that at least Java does so by using reflections and has lead to more then just a few massive security vulnerabilities.

Re: The Serde Rust Framework

#134

Earlier quoted context omitted.

The only downside is compile time bloat. Serde generates heaps and heaps of generic code. This all gets optimized away to be very efficient, but only once it reaches LLVM. Ever tried working on a crate with hundreds or thousands of de/serializable types? Compile times shoot through the roof really quickly, and serde is often the culprit. The maintainer of serde also created `miniserde` [1] to tackle this problem, whi…

Studies have shown that more than 87% of complaints about rust compile time are because people are enabling serde everywhere and for everything.

@burntsushi had a post where he explained that the common error-handling crates also add big compile-time costs. Most projects use them as well as serde, so the problem compounds.

Re: The Serde Rust Framework

#135
post #108

Earlier quoted context omitted.

Yes, because those same people seem to be unaware reflection is not the only way to use such APIs, revealing a superfical knowledge of how those ecosystems have been working for the last 25 years.

How is it implemented in Java or C# if not via runtime reflection? I've always assumed that's the way its done...

Java has annotation processors which can perform code generation, just like Serde does.

Manifold [1] an example framework that relies heavily on that, achieving similar things like Serde.

Java developers don't like to use these things too much though, unlike Rust developers who love their `#derive`, so you're mostly right that reflection-based serialization is still more common, but that's by choice.

Dependency Injection is in a similar situation: frameworks like Micronaut [2] can do it without reflection, and things like Google Dagger [3] have existed for several years that do the same thing... but still, most Spring Boot-based projects I know of use reflection-based DI as well... it's fast and the security issues have been largely mitigated nowadays.

[1] http://manifold.systems/

[2] https://docs.micronaut.io/latest/guide/

[3] https://rskupnik.github.io/dependency-injection-in-pet-proje...

Re: The Serde Rust Framework

#136
post #108

Earlier quoted context omitted.

Yes, because those same people seem to be unaware reflection is not the only way to use such APIs, revealing a superfical knowledge of how those ecosystems have been working for the last 25 years.

How is it implemented in Java or C# if not via runtime reflection? I've always assumed that's the way its done...

I don't program Java, but I have implemented and used similar things for C#.

Reflection is indeed used there, but not to serialize. It's only used once per type, to generate code.

C# has multiple ways to generate code in runtime. One is Reflection.Emit, allows to manually generate bytecode instructions + metadata. For instance, one can build new types in runtime. A typical pattern is implementing manually-written abstract class or interface with generated types: this way the manually-written code can call into runtime generated one.

Another method is System.Linq.Expressions. This one allows to generate code (no new types though, just functions) from expression trees, and provides API to build and transform these expression trees.

Regardless on the method, the generated code is no different from manually written code. JIT compiler can even inline things across, when generated code calls manually written one, or vice versa.

Re: The Serde Rust Framework

#137
post #100

Earlier quoted context omitted.

That has been the same in Java and .NET world for 20 years now, just plug the desired serializer from the standard library and be done with it.

Yay, except that at least Java does so by using reflections and has lead to more then just a few massive security vulnerabilities.

Just to be clear, there are Java libraries which do thinks better in a less problematic way, just the facilities shipped in the standard library are IMHO inadequate.

Re: The Serde Rust Framework

#138
post #124

Earlier quoted context omitted.

Option would look like tagged union in memory, so one bit "tag", 8 bytes u64 and padding. If type inside of Option has invalid values (like references which can't be null) one of these would be used for None, so Option is the same size as &T.

thanks and is the padding for aligned access (wasted space) or just full byte (poor performance on non x86/64 like architecture)? because that's the interesting part as to what's the tradeoff chosen and whether a dev can choose a different one. Any doc references are appreciated.

They are padded for aligned access, so on amd64 Option will be 16 bytes.[0] However, the unused discriminant values are counted as niches for enums wrapping that, so Option> is still 16 bytes (for example).

[0]: https://play.rust-lang.org/?version=stable&mode=debug&editio...

Re: The Serde Rust Framework

#139
post #2

serde is one of the best things about Rust in practice. It is more convenient to use serde to serialize/deserialize to some standard format like JSON or YAML than it is to write your own half-baked format and serialization/deserialization code --- even for the simplest tasks --- so you just stop doing the latter. This is a fundamental shift, and very good for your software. "Convenience" here actually covers a lot of…

The only downside is compile time bloat. Serde generates heaps and heaps of generic code. This all gets optimized away to be very efficient, but only once it reaches LLVM. Ever tried working on a crate with hundreds or thousands of de/serializable types? Compile times shoot through the roof really quickly, and serde is often the culprit. The maintainer of serde also created `miniserde` [1] to tackle this problem, whi…

In my observation, the Rust community is very hesitant to use dynamic dispatch, even where it wouldn't hurt performance (true for most things that involve I/O) and would dramatically reduce compile times.

I'm not sure why. The fact that Rust makes dyn traits significantly more verbose doesn't help, of course, but that can't be the only reason.

Re: The Serde Rust Framework

#140
post #102

Earlier quoted context omitted.

I’m pretty sure the non-null issue is fixable with a single config option, at least in GSON IIRC.

I don't believe it's fixable at all if you're using someone's third-party library that wasn't designed for GSON (unless something has changed a lot since I last used it). The annotation has to be stuck on the original field, AFAIK. And similar stuff apply to other invariants on the type. I believe these kinds of issues are somewhat fundamental when you decide you're going to start (de)serializing structures that neve…

In Jackson, you can use mixin classes to define annotations for 3rd party libraries.
Post reply on HN