Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Serde Rust Framework

serde.rs

Post reply on HN
Print view View original

111–120 of 159 posts

Re: The Serde Rust Framework

#111

Post by spoiler »

dwohnitmok wrote: post #85
spoiler wrote: post #63

Earlier quoted context omitted.

I don't think that's true. Or at least we have different ideas about what's being discussed... Let's take JSON or YAML for example: Rust's closest siblings C and C++ are typed, and you get a pretty much untyped messes when working with serialisation or deserialisation. You have to manually inspect every node or write manual ”NodeType" to struct conversation. They allow unwrapping to primitives at best (eg via templat…

> In Haskell it's a bit more awkward, but you can kinda pull something similar off in terms of ergonomics. I've not worked with JSON in Haskell that much that I need to look for it, but I don't remember there being something as ergonomic as serde. I think Haskell's most popular JSON library aeson is basically the same thing and probably a source of inspiration for serde. data MyCustomType = ... deriving (Generic, ToJ…

Ah, I've heard about aeson, but not used it. Thanks for pointing it out!

Re: The Serde Rust Framework

#112

Post by spoiler »

rudedogg wrote: post #77
spoiler wrote: post #63

Earlier quoted context omitted.

I don't think that's true. Or at least we have different ideas about what's being discussed... Let's take JSON or YAML for example: Rust's closest siblings C and C++ are typed, and you get a pretty much untyped messes when working with serialisation or deserialisation. You have to manually inspect every node or write manual ”NodeType" to struct conversation. They allow unwrapping to primitives at best (eg via templat…

> Of the "mainstream" languages Go is pretty much the only one that has as good ergonomics as Rust in that it supports it pretty much natively (via struct tags) Swift does this too, using the Codable Protocol. The compiler generates the protocol conformance code for you, if you say a type is `Codable`, and the properties the type contains are all Codable types (Int, Float, String, URL, etc. conform to it), you don't…

Ah, I don't use Apple devices, so I've never been able to play with Swift(1). So I didn't know Swift had it too; thank you for pointing it out!

(1): which is a shame, I remember when it came out that it seemed like a nice language

Re: The Serde Rust Framework

#113

Post by roca »

mwcampbell wrote: post #97
roca wrote: post #57

Earlier quoted context omitted.

Yeah, the format independence is amazing. In Pernosco we use bincode for IPC payloads and when we want to log IPCs we just serialize the payloads to JSON with zero developer effort.

Have you considered using a zero-copy serialization format like capnproto, Flat Buffers, or rkyv for binary IPC payloads? Or did you decide that getting that last increment of performance wasn't worth the trouble of using something less easy than serde?

Yeah our IPC marshalling just isn't that relevant for performance AFAIK.

Re: The Serde Rust Framework

#114

Post by spoiler »

vips7L wrote: post #73
spoiler wrote: post #63

Earlier quoted context omitted.

I don't think that's true. Or at least we have different ideas about what's being discussed... Let's take JSON or YAML for example: Rust's closest siblings C and C++ are typed, and you get a pretty much untyped messes when working with serialisation or deserialisation. You have to manually inspect every node or write manual ”NodeType" to struct conversation. They allow unwrapping to primitives at best (eg via templat…

Maybe we are having different ideas about what is being discussed, but in Java and C# (arguably the most mainstream languages) you have the same experience supported natively via annotations and attributes.

Ah, I stand corrected then.

I've worked with the Microsoft tech stack very very little, so I've never bothered to learn C#.

I've completely forgot about Java; I haven't used it in ages. Now that you mentioned Java, I realised Kotlin has it too.

Re: The Serde Rust Framework

#115

Post by spoiler »

pjmlp wrote: post #108
Jweb_Guru wrote: post #101

Earlier quoted context omitted.

It absolutely does not provide the same guarantees, and I've covered why in several posts now. Do you believe that people who disagree with you have never used any language other than Rust before? Having to deal with data structure invariants getting violated due to reflection is a huge headache and makes the parsers way less useful than they would be otherwise.

Yes, because those same people seem to be unaware reflection is not the only way to use such APIs, revealing a superfical knowledge of how those ecosystems have been working for the last 25 years.

How is it implemented in Java or C# if not via runtime reflection? I've always assumed that's the way its done...

Re: The Serde Rust Framework

#116

Post by the_mitsuhiko »

AaronO wrote: post #8

Whilst not classical de/serialization I wrote serde_v8 ( https://github.com/denoland/serde_v8 ), an expressive and ~maximally efficient bijection between v8 & rust. It has powered Deno's op-layer since 1.9 ( https://deno.com/blog/v1.9#faster-calls-into-rust-with-serde... ) and has enabled significant improvements in opcall overhead (close to 100x) whilst also simplifying said op-layer.

While what I am using it for is not for performance, but I'm also using Serde as a bridge for my template engine and it's such a nice experience. I just wish it was possible to pass auxiliary information between serializers without having to resort to thread locals.

Re: The Serde Rust Framework

#117

Post by mcintyre1994 »

bpicolo wrote: post #61
the_gipsy wrote: post #33

Earlier quoted context omitted.

Isn’t it insane to change your types to accommodate this? You have to change all fields accesses, it behaves differently, you lose non-nullability…

Don't you do that in rust too? You'd be working with Option instead of T.

Option is a really good way to describe a field that if present is an int, but may not be present, though. Pointer isn't.

Re: The Serde Rust Framework

#118

Post by Fiahil »

the_duke wrote: post #20
roca wrote: post #2

serde is one of the best things about Rust in practice. It is more convenient to use serde to serialize/deserialize to some standard format like JSON or YAML than it is to write your own half-baked format and serialization/deserialization code --- even for the simplest tasks --- so you just stop doing the latter. This is a fundamental shift, and very good for your software. "Convenience" here actually covers a lot of…

The only downside is compile time bloat. Serde generates heaps and heaps of generic code. This all gets optimized away to be very efficient, but only once it reaches LLVM. Ever tried working on a crate with hundreds or thousands of de/serializable types? Compile times shoot through the roof really quickly, and serde is often the culprit. The maintainer of serde also created `miniserde` [1] to tackle this problem, whi…

It's a tradeoff. Switch to Python and watch your serialization code become at least 10x slower at runtime.

Re: The Serde Rust Framework

#119

Post by [deleted] »

berkut wrote: post #44
pornel wrote: post #12

Earlier quoted context omitted.

The trick with Serde is that it decodes straight into a native Rust struct. You get most of validation for free (it also nicely takes advantage of Rust enums with data), and struct access is maximally fast. A generic JSON decoder would give you a dynamic structure that can contain anything, and then you'd have to pick it apart.

However, if you just want a very minimal subset of a JSON response (i.e. a 'status' string field out of say 20 other values in a map), is it not better to just look for one field, as opposed to have to create an entire struct in Rust to represent the full response? How does Serde work with optional extra fields? I know you can use Option , but that implies you know they exist - what happens if fields get added in the…

[deleted]

Re: The Serde Rust Framework

#120

Post by hawk_ »

apendleton wrote: post #22
the_gipsy wrote: post #13

I just came from rust to go and… what a disappointment. Missing fields default to some kind of default “zero value” - for any type, even full blown structs. You can’t tell the difference of a missing field or the field having the default value. So if a field has a validation of “must be greater than zero”, you can’t really give a proper error message. If user puts in “0” or omits the field, you always get a value of…

If you want a field to be able to be not-set, you just need to specify that by making it an `Option ` type. So like, if the field is a number, make it an `Option ` -- that way you can distinguish between not-set (`None`) and set to zero (`Some(0)`). Edit: whoops, misread which language it was you were frustrated with. Nevermind!

when Optional is used in rust what's the memory layout like? is the absence tracked using a pointer which is null or a bitvector of optional fields?
Post reply on HN