Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Serde Rust Framework

serde.rs

Post reply on HN
Print view View original

151–159 of 159 posts

Re: The Serde Rust Framework

#151

Post by Const-me »

spoiler wrote: post #145
Const-me wrote: post #136

Earlier quoted context omitted.

I don't program Java, but I have implemented and used similar things for C#. Reflection is indeed used there, but not to serialize. It's only used once per type, to generate code. C# has multiple ways to generate code in runtime. One is Reflection.Emit, allows to manually generate bytecode instructions + metadata. For instance, one can build new types in runtime. A typical pattern is implementing manually-written abs…

Ah the bytecode generation via Reflection.Emit seems like a really powerful feature. Does it feel "brittle" to use, or is it expressive enough to aid writing more robust code (to a degree, I realise it's not possible to aliminate every problem)? There's a weird (not in a bad way) elegance here that reminds me of lisp.

> Does it feel "brittle" to use

Yes and no.

No because when you try to do unsupported things like calling a method on an object which doesn’t support one, you gonna get an appropriate runtime exception.

Yes because if you fail lower-level things like local parameter allocation, you gonna get an appropriate runtime exception but that one is (1) too late, I’d prefer such things to be detected when you emit the code, not when trying to use the generated code (2) Lacks the context.

Overall, when I can I’m using that higher-level System.Linq.Expressions for runtime codegen. Things are much nicer at that level. I only using the low-level thing when I need to emit new types, like there: https://github.com/Const-me/ComLightInterop/blob/master/ComL...

Re: The Serde Rust Framework

#152

Post by dcow »

chakkepolja wrote: post #121
dcow wrote: post #107

Earlier quoted context omitted.

Right, serde is one of the many features about Rust that people miss when writing Go. Simply a group of downtrodden programmers bonding over a shared experience the memory of which evoked when reminiscing on a grand cathedral. But alas the mere reminder of the existence of such a grand structure weighs on the heart of the plebeian. No need to rub it in sorry gogrammers’ faces that their language is a limp noodle in c…

> No need to rub it in sorry gogrammers’ faces that their language is a limp noodle in comparison. Say what you want about Go, but gophers get better jobs than you, meanwhile almost all rust jobs are shady crypto startups. Hard truth lol.

I know how to write both, thank you very much. And I predominately use Rust and Swift at my current gig, which is not a crypto anything. Did you know Microsoft, Cloudflare, Google, and Mozilla all use Rust? Wow. Such shade. Much crypto. I guess you're right about them all being startups though.

Re: The Serde Rust Framework

#153

Post by ahahahahah »

brabel wrote: post #134
unanswered wrote: post #40

Earlier quoted context omitted.

Studies have shown that more than 87% of complaints about rust compile time are because people are enabling serde everywhere and for everything.

@burntsushi had a post where he explained that the common error-handling crates also add big compile-time costs. Most projects use them as well as serde, so the problem compounds.

I think that's a misunderstanding. That's a very different problem from the serde one.

The compile time costs for the error handling crates is due to increase in build graph size, not due to the cost of running the error handling proc macros or the cost of compiling the code generated from them. For anything but a small project, that increase in graph size is probably not a big deal (and likely the project is already using much of the same deps). This cost does not increase as a project gets larger.

serde also probably has the build graph size problem, but the bigger problem is the cost generating and compiling the generated code. This cost tends to increase as a project gets larger.

Re: The Serde Rust Framework

#154

Post by ahahahahah »

goldsteinq wrote: post #122
hawk_ wrote: post #120

Earlier quoted context omitted.

when Optional is used in rust what's the memory layout like? is the absence tracked using a pointer which is null or a bitvector of optional fields?

Option would look like tagged union in memory, so one bit "tag", 8 bytes u64 and padding. If type inside of Option has invalid values (like references which can't be null) one of these would be used for None, so Option is the same size as &T.

> If type inside of Option has invalid values

Can it actually use any general invalid bit pattern? I expect that it only supports an optimization for values that cannot be zero (and then it uses zero to indicate None).

Re: The Serde Rust Framework

#155

Post by the_duke »

codeflo wrote: post #139
the_duke wrote: post #20

Earlier quoted context omitted.

The only downside is compile time bloat. Serde generates heaps and heaps of generic code. This all gets optimized away to be very efficient, but only once it reaches LLVM. Ever tried working on a crate with hundreds or thousands of de/serializable types? Compile times shoot through the roof really quickly, and serde is often the culprit. The maintainer of serde also created `miniserde` [1] to tackle this problem, whi…

In my observation, the Rust community is very hesitant to use dynamic dispatch, even where it wouldn't hurt performance (true for most things that involve I/O) and would dramatically reduce compile times. I'm not sure why. The fact that Rust makes dyn traits significantly more verbose doesn't help, of course, but that can't be the only reason.

Writing `dyn` in front of types doesn't really make a difference, it's only a little bit of added verbosity.

The main problem is that trait objects are very limited.

There is no downcasting or upcasting (eg C++ dynamic_cast) and trait objects are limited to a single trait. You can't have `Box`.

That leads to lots of headaches in practice.

I hear downcasting might be on the horizon, so that's something to look forward to.

Re: The Serde Rust Framework

#156

Post by vlovich123 »

pjmlp wrote: post #149
vlovich123 wrote: post #142

Earlier quoted context omitted.

C++ has theoretical solutions to it. I’ve yet to see anyone actually do anything like that in any professional environment (from small startup codebase a to the largest c++ code bases that exist in the world) because that’s an unmaintainable solution with a high burden placed on the developer. Whatever the solution Rust comes up with I think will look at the practical component so as to actually see adoption. It seem…

We used to do such stuff at Nokia, on HP-UX with Clearcase and .o build caching supported by cleartool using aCC, while working on NetAct, professional enough?

My point was more meant to be that it's extremely rare, not that it doesn't ever happen. Do people partially specialize templates? I'm sure yes, it happens from time to time. Lots of things happen from time-to-time depending on the mix of the team & the problem domain. If you picked a random team/codebase, how many do you think are carefully optimizing partial template specialization? Moreover, what tools even exist to let you profile the contribution of a missing specialization and measure the impact of your change? Not sure how .o build caching relates to partial template specialization though.

With regards to your specific experience, I don't know that I'd say that Clearcase, HP-UX or aCC are main-stream professional C++ development environments. I think for that you're really looking at clang, gcc, & msvc with icc being a distant fourth that has itself moved to clang recently. The primary target OS would either be server Linux, Android, macOS, iOS with maybe FreeBSD picking up the slack. HP-UX has a niche but it's a small niche comparatively in terms of $ spent on developers in that ecosystem. In terms of source control, everyone is mostly on git or mercurial. Anyone not just refuses to get with the times because they have bought into their existing tool expertise but don't know how to adapt.

Re: The Serde Rust Framework

#157

Post by cztomsik »

I'm glad serde exists but I don't like HOW it works:

I think there should be a push for rust-lang to provide introspection. If you want to use serde, every crate needs to derive serde traits as well. This is not only bad architecture but it also prevents other libraries to get popular. For example, there are many "mini-serde" variants and I personally think, they are awesome (and surprisingly fast), yet you cannot practically use them because literally no crate supports anything else than serde.

https://github.com/dtolnay/miniserde

https://github.com/not-fl3/nanoserde

https://github.com/makepad/makepad/tree/master/render/micros...

Re: The Serde Rust Framework

#158

Post by jcelerier »

cztomsik wrote: post #157

I'm glad serde exists but I don't like HOW it works: I think there should be a push for rust-lang to provide introspection. If you want to use serde, every crate needs to derive serde traits as well. This is not only bad architecture but it also prevents other libraries to get popular. For example, there are many "mini-serde" variants and I personally think, they are awesome (and surprisingly fast), yet you cannot pr…

I wonder if the c++ approach of boost.pfr would be portable to rust ? It allows reflection on aggregates without needing to annotate anything: https://github.com/boostorg/pfr

Re: The Serde Rust Framework

#159

Post by cztomsik »

jcelerier wrote: post #158
cztomsik wrote: post #157

I'm glad serde exists but I don't like HOW it works: I think there should be a push for rust-lang to provide introspection. If you want to use serde, every crate needs to derive serde traits as well. This is not only bad architecture but it also prevents other libraries to get popular. For example, there are many "mini-serde" variants and I personally think, they are awesome (and surprisingly fast), yet you cannot pr…

I wonder if the c++ approach of boost.pfr would be portable to rust ? It allows reflection on aggregates without needing to annotate anything: https://github.com/boostorg/pfr

I think currently, the only closest thing is bevy-reflect. Which is nice, but IMHO it should be in the language because nobody supports it ATM and to get there, it will take at least the same time it took for serde to get supported.

https://crates.io/crates/bevy_reflect

Post reply on HN