This is really cool. But how does zero-copy deserialization work for &'a str with formats like JSON where the input data may have string escapes (and therefore needs to be mutated during deserialization)?
Serde 1.0.0 for Rust released
31–40 of 52 posts
Re: Serde 1.0.0 for Rust released
#32Earlier quoted context omitted.
I really really would love Avro support added... I'd do it if I had more Rust experience. The icing on the cake is if it could also interact with the Avro registry somehow.
What's Avro?
It allows you to define a schema, and then compile it to various languages and have multiple applications in multiple languages "speak" the same class, down to the binary level (for RPC, blob storage etc).
Re: Serde 1.0.0 for Rust released
#33Earlier quoted context omitted.
I really really would love Avro support added... I'd do it if I had more Rust experience. The icing on the cake is if it could also interact with the Avro registry somehow.
What's Avro?
It's not as popular as Protocol Buffers, Thrift, MessagePack etc., probably because it started out in Java land (I believe it came from Doug Nutch of Lucene fame).
I've never used it, but one benefit of Avro is that it can embed the schema in the serialized output, which allows clients to consume data even though they don't have the IDL, which isn't possible with Protobuf and Thrift. MessagePack does include types and map keys, but doesn't have a schema.
Re: Serde 1.0.0 for Rust released
#34This is really cool. But how does zero-copy deserialization work for &'a str with formats like JSON where the input data may have string escapes (and therefore needs to be mutated during deserialization)?
It doesn't. You can use Cow if you want string escapes to work, but then it is not really zero-copy anymore.
Re: Serde 1.0.0 for Rust released
#35I really love Serde, but one consequence of splitting out the formats into different libraries is that different formats can be of substantially different levels of quality. My impression is that JSON support is best-of-class, but I have no idea about the others. For example, I've been looking at the CBOR library for Serde [1], and it's not obvious whether the library is full-featured, robust, actively supported, etc…
This is a global problem for packages, and I think there's a push for a feature where developers can mark for a crate how stable it is. Anyways for the packages without passing tests we already know it can't be stable yet, as it doesn't even have unit test passing guarantee :)
Re: Serde 1.0.0 for Rust released
#36Serde really is one of the gems of the Rust ecosystem. It is a (de)serialization framework that can be quite easily implemented for various serialization formats like JSON, MessagePack, Yaml, toml, ... It enables automatic and very performant (de)serialization of you data, into different formats. Often with a simple: ` #[derive(Serialize, Deserialize)] struct Data { ... } `
I really really would love Avro support added... I'd do it if I had more Rust experience. The icing on the cake is if it could also interact with the Avro registry somehow.
I keep picking this up to try and help and then dropping it. I wish I had more time in the day.
It looks like deserialization is mostly there, but not serialization.
Re: Serde 1.0.0 for Rust released
#37Earlier quoted context omitted.
What's Avro?
It's not as popular as Protocol Buffers, Thrift, MessagePack etc., probably because it started out in Java land (I believe it came from Doug Nutch of Lucene fame). I've never used it, but one benefit of Avro is that it can embed the schema in the serialized output, which allows clients to consume data even though they don't have the IDL, which isn't possible with Protobuf and Thrift. MessagePack does include types an…
s/Doug Nutch/Doug Cutting/
Re: Serde 1.0.0 for Rust released
#38Earlier quoted context omitted.
It's not as popular as Protocol Buffers, Thrift, MessagePack etc., probably because it started out in Java land (I believe it came from Doug Nutch of Lucene fame). I've never used it, but one benefit of Avro is that it can embed the schema in the serialized output, which allows clients to consume data even though they don't have the IDL, which isn't possible with Protobuf and Thrift. MessagePack does include types an…
s/Doug Nutch/Doug Cutting/
Oops. That should have been "Doug Cutting of Nutch and Lucene fame".
Re: Serde 1.0.0 for Rust released
#39This is really cool. But how does zero-copy deserialization work for &'a str with formats like JSON where the input data may have string escapes (and therefore needs to be mutated during deserialization)?
It doesn't. You can use Cow if you want string escapes to work, but then it is not really zero-copy anymore.
So, what, if you use &str you get the raw data, escapes and all? That's not very good, especially because it means round-tripping your struct through JSON can return the wrong result.
I assume that Cow will only produce an owned string if there are strong escapes that need decoding? If so, that's probably the best approach for decoding appropriately, as you'd get zero-copy as long as no mutations are required, but round-tripping through JSON would still work right.
Re: Serde 1.0.0 for Rust released
#40As an aside: > Rust's "orphan rule" allows writing a trait impl only if either your crate defines the trait or defines one of the types the impl is for. That means if your code uses a type defined outside of your crate without a Serde impl, you resort to newtype wrappers or other obnoxious workarounds for serializing it. I hadn't realized this. The justification is reasonable - preventing ambiguity when resolving tra…
Yeah, in Haskell it's allowed but outputs a warning. So a lot of my modules use -fno-warn-orphans :D
The reason is that class instances are really not first-class in Haskell. You can never import a module and choose not to import its instances, not even with
import Module (just_one_symbol_i_need)
import Module (instance toJsonText, ...)
PureScript as a "Haskell without the mistakes" really is a great idea.