Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Serde Rust Framework

serde.rs

Post reply on HN
Print view View original

91–100 of 159 posts

Re: The Serde Rust Framework

#91

Post by thedracle »

UK-Al05 wrote: post #4
roca wrote: post #2

serde is one of the best things about Rust in practice. It is more convenient to use serde to serialize/deserialize to some standard format like JSON or YAML than it is to write your own half-baked format and serialization/deserialization code --- even for the simplest tasks --- so you just stop doing the latter. This is a fundamental shift, and very good for your software. "Convenience" here actually covers a lot of…

I think in most ecosystems it's easier to use a prebuilt serializer/deserializer than your own format.

At least in languages with similar native performance as Rust (C/C++), in my personal experience, serialization has often been difficult and manual, and most libraries have fallen short of my expectations, or had serious performance, security, or toolchain issues, to the point I usually hand-serialize myself.

Re: The Serde Rust Framework

#92

Post by munro »

the_duke wrote: post #20
roca wrote: post #2

serde is one of the best things about Rust in practice. It is more convenient to use serde to serialize/deserialize to some standard format like JSON or YAML than it is to write your own half-baked format and serialization/deserialization code --- even for the simplest tasks --- so you just stop doing the latter. This is a fundamental shift, and very good for your software. "Convenience" here actually covers a lot of…

The only downside is compile time bloat. Serde generates heaps and heaps of generic code. This all gets optimized away to be very efficient, but only once it reaches LLVM. Ever tried working on a crate with hundreds or thousands of de/serializable types? Compile times shoot through the roof really quickly, and serde is often the culprit. The maintainer of serde also created `miniserde` [1] to tackle this problem, whi…

I've been using Rust for just a little bit, building applications/services/scrapers/databases/forking libraries, but I haven't noticed these issues--the biggest compile time issue I had was including RocksDB. I'm trying to imagine what you're building. :} 99% of the time I'm just doing incremental debug builds, and haven't really minded the compile times when doing release builds. Overall it everything has been fast, to the point where I want to start scripting with Rust.

Re: The Serde Rust Framework

#93

Post by Yacoby »

Jweb_Guru wrote: post #83
UK-Al05 wrote: post #66

Earlier quoted context omitted.

I rarely have to think about deserialisation in c# or java either...

I wrote a lot of Java back in the day. I elaborated on why I disagree in the other comment. I do not know if the situation in C# is similar.

For context, the example from the Serde wiki written in Java using Jackson

  public record Point(int x, int y) {}

  // [...]

  var objectMapper = new ObjectMapper();
  var jsonString = objectMapper.writeValueAsString(new Point(2,1));
It isn't obvious to me what advantage Serde offers that you couldn't get with Jackson or other similar libraries in Java (although I get that Java and Rust are different languages, and that there may not be something this ergonomic in the likes of C++)

Re: The Serde Rust Framework

#94

Post by Wulf_ »

option_greek wrote: post #89
_xyub wrote: post #80

Earlier quoted context omitted.

I thought I'd share this project in case anyone wanted to quickly evaluate rust for web https://github.com/wulf/create-rust-app Although there's a lot of work left to be done, I'd love to hear feedback, painpoints and ideas you have :) Hoping to have documentation up at create-rust-app.dev soon~

Please consider giving warp as choice for the web server.

Any particular reason?

Re: The Serde Rust Framework

#95

Post by masklinn »

berkut wrote: post #44
pornel wrote: post #12

Earlier quoted context omitted.

The trick with Serde is that it decodes straight into a native Rust struct. You get most of validation for free (it also nicely takes advantage of Rust enums with data), and struct access is maximally fast. A generic JSON decoder would give you a dynamic structure that can contain anything, and then you'd have to pick it apart.

However, if you just want a very minimal subset of a JSON response (i.e. a 'status' string field out of say 20 other values in a map), is it not better to just look for one field, as opposed to have to create an entire struct in Rust to represent the full response? How does Serde work with optional extra fields? I know you can use Option , but that implies you know they exist - what happens if fields get added in the…

> However, if you just want a very minimal subset of a JSON response (i.e. a 'status' string field out of say 20 other values in a map), is it not better to just look for one field, as opposed to have to create an entire struct in Rust to represent the full response?

You can define a struct with just the status field.

> How does Serde work with optional extra fields? I know you can use Option, but that implies you know they exist - what happens if fields get added in the future silently? I guess the schema changes then, which isn't good, but that might happen?

Unknown fields are skipped by default.

Re: The Serde Rust Framework

#96

Post by shp0ngle »

the_gipsy wrote: post #33
X6S1x6Okd1st wrote: post #29

Earlier quoted context omitted.

I've usually seen this handled by having everything be a pointer, the default value of the pointer is nil. You can also use the SQL values.

Isn’t it insane to change your types to accommodate this? You have to change all fields accesses, it behaves differently, you lose non-nullability…

Well that's go for you.

Go is simple and "default case" which is simple and works for most cases... and then you have edge cases where you need to bend over backwards.

As an exercise, try to use comma (",") in JSON field name.

Re: The Serde Rust Framework

#97

Post by mwcampbell »

roca wrote: post #57
pornel wrote: post #17

Because Serde is a common API for pretty much all serialisation formats supported in Rust, it's very easy to try multiple formats to choose the best size/speed you need. JSON too big? You can make it CBOR or Msgpack with a couple of lines. Want faster? Call bincode instead.

Yeah, the format independence is amazing. In Pernosco we use bincode for IPC payloads and when we want to log IPCs we just serialize the payloads to JSON with zero developer effort.

Have you considered using a zero-copy serialization format like capnproto, Flat Buffers, or rkyv for binary IPC payloads? Or did you decide that getting that last increment of performance wasn't worth the trouble of using something less easy than serde?

Re: The Serde Rust Framework

#98

Post by Jweb_Guru »

Yacoby wrote: post #93
Jweb_Guru wrote: post #83

Earlier quoted context omitted.

I wrote a lot of Java back in the day. I elaborated on why I disagree in the other comment. I do not know if the situation in C# is similar.

For context, the example from the Serde wiki written in Java using Jackson public record Point(int x, int y) {} // [...] var objectMapper = new ObjectMapper(); var jsonString = objectMapper.writeValueAsString(new Point(2,1)); It isn't obvious to me what advantage Serde offers that you couldn't get with Jackson or other similar libraries in Java (although I get that Java and Rust are different languages, and that ther…

For that example there's no advantage. But it's really, really easy to find examples where there are (the simplest example is, anything with pointers with @NonNull annotations, or anything which is performance-sensitive and contains other objects).

Re: The Serde Rust Framework

#99

Post by pjmlp »

vlovich123 wrote: post #69
johnfn wrote: post #43

Earlier quoted context omitted.

These concerns are valid. Data serialization/deserialization is an extremely standard thing to need to be able to do. It's not the user's fault if they use the obvious tool for an obvious job and suddenly have bloated compile times.

Seems like there’s opportunity for tighter integration with the compiler where serde moves into the compiler or there are acceleration primitives rather than doing the AST macros thing. This sounds similar to the problems of c++ template bloat.

Except C++ template bloat has solutions not yet available in Rust.

For example you can pre-instatiate the most common type parameters and just stuck those instantiations into a binary library.

Also if you have a nice C++ compiler environment like Visual C++, you can even do edit-and-continue for many kinds of changes, and VS 2022 will double down on that capability.

Re: The Serde Rust Framework

#100

Post by pjmlp »

Jweb_Guru wrote: post #35
UK-Al05 wrote: post #4

Earlier quoted context omitted.

I think in most ecosystems it's easier to use a prebuilt serializer/deserializer than your own format.

Nah, not like it is in Rust. For the vast majority of projects, serde makes parsing / serializing something they don't even have to think about , without having to sacrifice type safety to get there. Until you experience that in a large project with a lot of moving parts and developers, it's hard to appreciate just how many developer cycles you were wasting before on stuff that can be totally automated. It's format-a…

That has been the same in Java and .NET world for 20 years now, just plug the desired serializer from the standard library and be done with it.
Post reply on HN