Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serde 1.0.0 for Rust released

github.com

Post reply on HN
Print view View original

41–50 of 52 posts

Re: Serde 1.0.0 for Rust released

#41

Post by conradev »

eridius wrote: post #39
whyever wrote: post #34

Earlier quoted context omitted.

It doesn't. You can use Cow if you want string escapes to work, but then it is not really zero-copy anymore.

So, what, if you use &str you get the raw data, escapes and all? That's not very good, especially because it means round-tripping your struct through JSON can return the wrong result. I assume that Cow will only produce an owned string if there are strong escapes that need decoding? If so, that's probably the best approach for decoding appropriately, as you'd get zero-copy as long as no mutations are required, but ro…

That makes sense. I want to rewrite my plist parser to both use Serde and be zero copy (https://github.com/conradev/plist-rs/issues), and for binary plists you would similarly need to convert UTF-16 strings to UTF-8, but wouldn't need to touch UTF-8 strings.

I wonder how the API would work. Would json::Value be modified to have a lifetime and contain Cow enums? How would it implement ToOwned? It almost seems like there would have to be separate types, a json::Value and json::OwnedValue.

Re: Serde 1.0.0 for Rust released

#42

Post by steveklabnik »

vvanders wrote: post #27
steveklabnik wrote: post #26

Earlier quoted context omitted.

Active work is going into integer generics, which will solve this issue. It has taken a while but isn't exactly simple either!

Glad to hear it! I guess it's just jarring since the rest of Rust is so well crafted it felt really abrupt to run up against the 32 length fixed size limits. It's also somewhat annoying to have to coerce it into a slice via &array[..] instead of &array. It's workable and when I finally get the library to a stable state I definitely plan to share what went well and where I saw pain points.

Yup, totally hear you. It's totally achievable, but not simple; Haskell has this issue too, for example. There's been a few iterations of the design already. The current approach is to cut scope as much as possible to ship the basics sooner.

Re: Serde 1.0.0 for Rust released

#43

Post by stevedonovan »

geofft wrote: post #9

I'm working on an strace implementation in Rust, with an enum (tagged union) for system calls: enum Syscall { Open { pathname: Buffer, flags: u64, count: u64 }, Read { fd: u64, buf: Buffer, count: u64 }, ... } I was very pleasantly surprised at being able to add a couple of dependencies, add #[derive(Serialize)] right above the struct, change two lines of my main driver program, and get an strace --json with useful o…

Yes, and sometimes you really pay for runtime reflection - I've seen Java programs reduced to Python performance by excessive use of reflection. Whereas, this is _compile-time reflection_!

Re: Serde 1.0.0 for Rust released

#45

Post by RcouF1uZ4gsC »

steveklabnik wrote: post #42
vvanders wrote: post #27

Earlier quoted context omitted.

Glad to hear it! I guess it's just jarring since the rest of Rust is so well crafted it felt really abrupt to run up against the 32 length fixed size limits. It's also somewhat annoying to have to coerce it into a slice via &array[..] instead of &array. It's workable and when I finally get the library to a stable state I definitely plan to share what went well and where I saw pain points.

Yup, totally hear you. It's totally achievable, but not simple; Haskell has this issue too, for example. There's been a few iterations of the design already. The current approach is to cut scope as much as possible to ship the basics sooner.

It is this lack of non-type template parameters, template templates (ie higher minded types), and variadics that make Rust feel like a step backwards compared to C++. When, Rust gets these, it will remove a a major blocker for me.

Re: Serde 1.0.0 for Rust released

#46

Post by ktta »

emmelaich wrote: post #44
ktta wrote: post #22

There's also capnproto-rust[1] if anyone was wondering. [1]: https://github.com/dwrensha/capnproto-rust

... but it doesn't use Serde. Should it? Or can it? Am curious, not a criticism.

They target different things. Serde is something that simply serializes data into common text based or binary based format.

But there's been some innovation done to make that even more efficient. It started with protocol buffers[1]. So you can then basically write .proto files which are based on protocol buffers' own schema[2] which look like this[3]. What's special about these schemas is that they can be strongly typed, and then after a schema is written, which is a .proto file (in case of protocol buffers), code for any language can be generated to receive and parse the binary encoded message properly, with proper error checking. This avoids re-writing code in different languages if a RPC protocol is changed. It also offers other advantages and you can look into the docs for that.

Then, the author of protocol buffers left Google and created something called Capnproto[4], which improved on it in many ways. Now, what I linked to is a rust program supporting capnproto's own schema[5]

[1]: https://developers.google.com/protocol-buffers/

[2]: https://developers.google.com/protocol-buffers/docs/proto3

[3]: https://github.com/WhisperSystems/libsignal-protocol-c/blob/...

[4]: https://capnproto.org/

[5]: https://capnproto.org/language.html

Re: Serde 1.0.0 for Rust released

#47

Post by whyever »

eridius wrote: post #39
whyever wrote: post #34

Earlier quoted context omitted.

It doesn't. You can use Cow if you want string escapes to work, but then it is not really zero-copy anymore.

So, what, if you use &str you get the raw data, escapes and all? That's not very good, especially because it means round-tripping your struct through JSON can return the wrong result. I assume that Cow will only produce an owned string if there are strong escapes that need decoding? If so, that's probably the best approach for decoding appropriately, as you'd get zero-copy as long as no mutations are required, but ro…

Yes, I don't think you can do better if forced to use &str. The point of using Cow is indeed to only copy when necessary like you assumed.

You could have zero-copy roundtrips if you used escapes in the deserialized strings as well.

Re: Serde 1.0.0 for Rust released

#48

Post by steveklabnik »

stusmall wrote: post #29
steveklabnik wrote: post #16

Earlier quoted context omitted.

https://crates.io/crates/serde_protobuf (I haven't tried it)

From the README: "Serialization is not yet implemented in this version."

We really have to start rendering READMEs on crates.io...

Re: Serde 1.0.0 for Rust released

#50

Post by emmelaich »

ktta wrote: post #46
emmelaich wrote: post #44

Earlier quoted context omitted.

... but it doesn't use Serde. Should it? Or can it? Am curious, not a criticism.

They target different things. Serde is something that simply serializes data into common text based or binary based format. But there's been some innovation done to make that even more efficient. It started with protocol buffers[1]. So you can then basically write .proto files which are based on protocol buffers' own schema[2] which look like this[3]. What's special about these schemas is that they can be strongly ty…

I'm aware of the details of Cap'n Proto.

But I thought since Avro is somewhat similar to capnproto and it uses Serde (in Rust) then capnproto could/should too.

But it sounds like it is a "should not"

Post reply on HN