Live data from Hacker News

Serde 1.0.0 for Rust released

github.com

11–20 of 52 posts

Re: Serde 1.0.0 for Rust released

#11
post #6

Very nice: > Zero-copy deserialization > […] The semantics of Rust guarantee that the input data outlives the period during which the output struct is in scope, meaning it is impossible to have dangling pointer errors as a result of losing the input data while the output struct still refers to it.

>The semantics of Rust guarantee that the input data outlives the period during which the output struct is in scope To be fair, so does the semantics of every language with garbage collection -- keeping things alive while there are references to them is the bread and butter of GC. EDIT: I do think it's impressive that Rust can manage this without the overhead of GC. But the sentence from the release notes immediately…

No, I think Rust still does this uniquely in a way that isn't available with a GC language. Imagine the following design (assume fd is a UDP socket or something, and so each read returns a complete message):

    char buf[1024];
    while (read(fd, buf, 1024)) {
        messages.push_back(deserialize(buf));
    }

    for (message: messages) {
        print(message);
    }
Garbage collection will keep buf alive, but won't guarantee that buf isn't being mutated while it's alive. Rust's ownership system will guarantee that. In Rust, the read() function would require a mutable (i.e., unique) reference to the buffer, and serde's deserialization function also requires a reference to the buffer, preventing read() from being callable while the deserialized objects continue to exist.

I think doing reader/writer refcounting at runtime is hard because this is a case where there's nothing reasonable to do at runtime if you have incompatible references. At best you can do copy-on-write, but then you silently lose the zero-copy performance. You really want a compile-time error saying "You structured this code wrong, go redesign it or add some copies."

Re: Serde 1.0.0 for Rust released

#12
post #4

As an aside: > Rust's "orphan rule" allows writing a trait impl only if either your crate defines the trait or defines one of the types the impl is for. That means if your code uses a type defined outside of your crate without a Serde impl, you resort to newtype wrappers or other obnoxious workarounds for serializing it. I hadn't realized this. The justification is reasonable - preventing ambiguity when resolving tra…

Luckily, because of rust's "newtype" types and the Deref trait, wrapping isn't too much of a pain.

Re: Serde 1.0.0 for Rust released

#13
post #6

Very nice: > Zero-copy deserialization > […] The semantics of Rust guarantee that the input data outlives the period during which the output struct is in scope, meaning it is impossible to have dangling pointer errors as a result of losing the input data while the output struct still refers to it.

>The semantics of Rust guarantee that the input data outlives the period during which the output struct is in scope To be fair, so does the semantics of every language with garbage collection -- keeping things alive while there are references to them is the bread and butter of GC. EDIT: I do think it's impressive that Rust can manage this without the overhead of GC. But the sentence from the release notes immediately…

I think the point here is that the source data might be mutated, without you touching the deserialized struct. Rust prevents that from becoming a problem.

Re: Serde 1.0.0 for Rust released

#14
post #4

As an aside: > Rust's "orphan rule" allows writing a trait impl only if either your crate defines the trait or defines one of the types the impl is for. That means if your code uses a type defined outside of your crate without a Serde impl, you resort to newtype wrappers or other obnoxious workarounds for serializing it. I hadn't realized this. The justification is reasonable - preventing ambiguity when resolving tra…

Yeah, in Haskell it's allowed but outputs a warning. So a lot of my modules use -fno-warn-orphans :D

Re: Serde 1.0.0 for Rust released

#17
post #6

Very nice: > Zero-copy deserialization > […] The semantics of Rust guarantee that the input data outlives the period during which the output struct is in scope, meaning it is impossible to have dangling pointer errors as a result of losing the input data while the output struct still refers to it.

>The semantics of Rust guarantee that the input data outlives the period during which the output struct is in scope To be fair, so does the semantics of every language with garbage collection -- keeping things alive while there are references to them is the bread and butter of GC. EDIT: I do think it's impressive that Rust can manage this without the overhead of GC. But the sentence from the release notes immediately…

I think the uniquely Rust bit is the memory efficiency. Instead of allocating memory for the raw data and the deserialized memory, you can simply reference the raw data directly through a typed variable, if I read this correctly. I'd assume a garbage collector does really help with this kind of efficiency. The cost here is compile time, but for performance critical code, it might well be a good trade off.

Re: Serde 1.0.0 for Rust released

#18
post #4

As an aside: > Rust's "orphan rule" allows writing a trait impl only if either your crate defines the trait or defines one of the types the impl is for. That means if your code uses a type defined outside of your crate without a Serde impl, you resort to newtype wrappers or other obnoxious workarounds for serializing it. I hadn't realized this. The justification is reasonable - preventing ambiguity when resolving tra…

This is true, however you can get around a lot of the annoyance of it with conversion traits like From and Into.

It still takes a bit of boilerplate to write unfortunately.

The good news is this likely won't be the case forever, it looks like specialization (I think it's called) and some other type system features will allow you to write some code to resolve this. Intersection impl's I think it's called. Correct me if I'm wrong anyone.

Re: Serde 1.0.0 for Rust released

#19
post #3

Serde really is one of the gems of the Rust ecosystem. It is a (de)serialization framework that can be quite easily implemented for various serialization formats like JSON, MessagePack, Yaml, toml, ... It enables automatic and very performant (de)serialization of you data, into different formats. Often with a simple: ` #[derive(Serialize, Deserialize)] struct Data { ... } `

I really really would love Avro support added... I'd do it if I had more Rust experience. The icing on the cake is if it could also interact with the Avro registry somehow.

Re: Serde 1.0.0 for Rust released

#20
post #19
post #3

Serde really is one of the gems of the Rust ecosystem. It is a (de)serialization framework that can be quite easily implemented for various serialization formats like JSON, MessagePack, Yaml, toml, ... It enables automatic and very performant (de)serialization of you data, into different formats. Often with a simple: ` #[derive(Serialize, Deserialize)] struct Data { ... } `

I really really would love Avro support added... I'd do it if I had more Rust experience. The icing on the cake is if it could also interact with the Avro registry somehow.

What's Avro?
Post reply on HN