Live data from Hacker News

The Serde Rust Framework

serde.rs

61–70 of 159 posts

Re: The Serde Rust Framework

#61

Earlier quoted context omitted.

I've usually seen this handled by having everything be a pointer, the default value of the pointer is nil. You can also use the SQL values.

Isn’t it insane to change your types to accommodate this? You have to change all fields accesses, it behaves differently, you lose non-nullability…

Don't you do that in rust too? You'd be working with Option instead of T.

Re: The Serde Rust Framework

#62
post #34

Earlier quoted context omitted.

Isn’t it insane to change your types to accommodate this? You have to change all fields accesses, it behaves differently, you lose non-nullability…

> Isn’t it insane to change your types to accommodate this? Welcome to Go! This was a real pain for us at my last job, where we used go-swagger to generate REST API code. All the generated structs had pointers for _non-optional_ fields so it could also check that they were populated. Meanwhile, optional fields were not pointers because they could use the default value. So then we ended up with lots of code doing chec…

Pretty excited for Go 1.18 for that reason! Will likely take some time for the ecosystem to shuffle out good solutions for different bits, though

Re: The Serde Rust Framework

#63
post #37
post #16

Earlier quoted context omitted.

Yea, but with serde, you don't get an untyped mess. You either get the type you expected or you get an error.

Same is true in any typed language.

I don't think that's true. Or at least we have different ideas about what's being discussed...

Let's take JSON or YAML for example:

Rust's closest siblings C and C++ are typed, and you get a pretty much untyped messes when working with serialisation or deserialisation. You have to manually inspect every node or write manual ”NodeType" to struct conversation. They allow unwrapping to primitives at best (eg via template specialisation).

In Haskell it's a bit more awkward, but you can kinda pull something similar off in terms of ergonomics. I've not worked with JSON in Haskell that much that I need to look for it, but I don't remember there being something as ergonomic as serde.

Typescript doesn't support this type of validation either out of the box (there might be tools that add validation).

In Elixir there's the Poison library, or it could be done with Kernel.struct/2 (not 100% sure this will work though).

Of the "mainstream" languages Go is pretty much the only one that has as good ergonomics as Rust in that it supports it pretty much natively (via struct tags)

---

This list excludes codegen tools which can generate (de)serialisers from an external schema (a la capnp, grpc, jsonschema, etc).

Re: The Serde Rust Framework

#64
post #4

Earlier quoted context omitted.

I think in most ecosystems it's easier to use a prebuilt serializer/deserializer than your own format.

Nah, not like it is in Rust. For the vast majority of projects, serde makes parsing / serializing something they don't even have to think about , without having to sacrifice type safety to get there. Until you experience that in a large project with a lot of moving parts and developers, it's hard to appreciate just how many developer cycles you were wasting before on stuff that can be totally automated. It's format-a…

These sorts of things are really convenient, but the idea of tightly binding your type definitions to serialization formats always makes be a little nervous: I haven’t used Rust enough in practice to have an opinion here, but I’m wary of a solution in this space that makes it too easy to ignore the forward/backwards compatibility of your external format.

Re: The Serde Rust Framework

#65

Earlier quoted context omitted.

Nah, not like it is in Rust. For the vast majority of projects, serde makes parsing / serializing something they don't even have to think about , without having to sacrifice type safety to get there. Until you experience that in a large project with a lot of moving parts and developers, it's hard to appreciate just how many developer cycles you were wasting before on stuff that can be totally automated. It's format-a…

These sorts of things are really convenient, but the idea of tightly binding your type definitions to serialization formats always makes be a little nervous: I haven’t used Rust enough in practice to have an opinion here, but I’m wary of a solution in this space that makes it too easy to ignore the forward/backwards compatibility of your external format.

Yeah, for sure that is something to be cautious of. For cases where I expect backwards compatibility to be a concern, I will usually make the top-level type of the message or format an enum with each variant being a version (initially with just a single version, V1 or V0). That way, if we need to make backwards-incompatible format changes, we can create a new variant, and potentially even automatically migrate between formats (I've done this a few times). Then all we have to do is call a function to migrate to the latest format, and we can use whatever that is for all the actual program code (generally it will be a type alias for the version structure of the same name). Of course, if this gets too complex or you start to have non-treelike relationships between values, you still want to fall back on a more traditional database schema, but this kind of versioning can get you a long way.

That said, it's incredibly refreshing to be able to focus on these semantic and migration concerns without having to worry about how to deal with parsing them later. serde doesn't solve enforcing backwards compatibility for you, but it sure makes doing the right thing a lot easier. You never have to worry about some code accidentally forgetting to check the version field, people carelessly mixing and matching messages with different versions, or fields that "should" never be set but are there for backwards compatibility. That's worth a lot to me!

Re: The Serde Rust Framework

#66
post #4

Earlier quoted context omitted.

I think in most ecosystems it's easier to use a prebuilt serializer/deserializer than your own format.

Nah, not like it is in Rust. For the vast majority of projects, serde makes parsing / serializing something they don't even have to think about , without having to sacrifice type safety to get there. Until you experience that in a large project with a lot of moving parts and developers, it's hard to appreciate just how many developer cycles you were wasting before on stuff that can be totally automated. It's format-a…

I rarely have to think about deserialisation in c# or java either...

Re: The Serde Rust Framework

#67

Earlier quoted context omitted.

You misunderstood, parent is complaining about Go, not about Rust. In Go you would have to use a pointer to represent Option , which makes code really awkward. Go serialization really has quite a few issues apart from default initialization. Configuration by somewhat weird struct tag strings which are only evaluated/validated at runtime, de/serialization is all done via reflection (unless you want to use code generat…

> In Go you would have to use a pointer to represent Option , which makes code really awkward. This sounded surprising to me, then I remembered Go doesn’t have generics. I imagine this won’t be as much of an issue when it does?

Perhaps if they were to remove the concept of nil in golang, but since they are all about backwards compatibility (which is nice to be fair), I don't think there's much change of seeing mainstream adoption because of how many existing projects will rely on default value in certain circumstances

I'd love to be proven wrong, but even with generics if you start using option types, it's going to be like JS and TypeScript where you'll have some parts nicely typed and others are the wild west

Re: The Serde Rust Framework

#68
post #61

Earlier quoted context omitted.

Isn’t it insane to change your types to accommodate this? You have to change all fields accesses, it behaves differently, you lose non-nullability…

Don't you do that in rust too? You'd be working with Option instead of T.

The critical difference is that an Option is not a T, but a nil pointer is still a pointer.

The stylistic difference is that the Rust type is semantic: your optional values are Options. The Go type is silly: why does being a pointer (which is allowed to be nil) imply required?

Re: The Serde Rust Framework

#69
post #43

Earlier quoted context omitted.

Studies have shown that more than 87% of complaints about rust compile time are because people are enabling serde everywhere and for everything.

These concerns are valid. Data serialization/deserialization is an extremely standard thing to need to be able to do. It's not the user's fault if they use the obvious tool for an obvious job and suddenly have bloated compile times.

Seems like there’s opportunity for tighter integration with the compiler where serde moves into the compiler or there are acceleration primitives rather than doing the AST macros thing. This sounds similar to the problems of c++ template bloat.
Post reply on HN