Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

greptime.com

Post reply on HN
Print view View original

31–40 of 63 posts

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#31

Post by sapiogram »

riku_iki wrote: post #19
wokwokwok wrote: post #14

Earlier quoted context omitted.

While it’s a nice bed time story if you like go, the reality is no, it’s not typical. Go has a good perf story, but typically rust or c++ would be faster after heavy optimisation; and should be more or less on par with typical applications. This isn’t a critique of go, and shouldn’t surprise anyone. Typically go also has unexpected optimisation hoops to jump through and problems related to the heavy use of channels (…

> but typically rust or c++ would be faster after heavy optimisation; it is concerning how much digging was required to optimize rust code in this case.

The Go code is already hyper-optimized by experts, just not by the blog authors, so you don't read about it here. As someone who has tried to write high-performance Go code on occasion, I can assure you that a ton of digging would have been required on that side as well.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#32

Post by reacharavindh »

netprole wrote: post #17
reacharavindh wrote: post #12

Earlier quoted context omitted.

Exactly, then why does it matter that the author had anything in their post as a figure of speech or analogy? Is it wrong to post a meme of a dog sitting near fire - https://knowyourmeme.com/memes/this-is-fine As a joke from SREs who handle firefighting calls? Does it offend dog lovers, people who are scared of fire?

I don't think I understand the comparison. Can you think of another one that involves a weapon and directed violence?

It’s simple. You ignore fire as a bad thing and don’t even consider that as equivalent comparison to a knife. But someone who was affected by arson probably will say “why include fire in a blog post?!” . My point is, it is not on the author to think about all these effect when they write/speak. As a mature society, we should learn to not expect every source to filter their thoughts and rather expect the consumer to filter out what they don’t want.

By not valuing fire/arson at the same level of concern as Knife(important to you), you just validated the core of the problem.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#33

Post by pjmlp »

wokwokwok wrote: post #14
MrBuddyCasino wrote: post #11

So in this case, with some modest performance engineering, Golang is surprisingly fast out of the box, and Rust requires more effort and increasingly less idiomatic code to reach the same result. Is this typical?

While it’s a nice bed time story if you like go, the reality is no, it’s not typical. Go has a good perf story, but typically rust or c++ would be faster after heavy optimisation; and should be more or less on par with typical applications. This isn’t a critique of go, and shouldn’t surprise anyone. Typically go also has unexpected optimisation hoops to jump through and problems related to the heavy use of channels (…

While I am not a big Go fan, most people validating Go's performance mix language and implementation, and forget gccgo also exists, sharing many optimizations offered by GCC's backend.

Unfortunely, it seems stuck in Go 1.18, the last pre-generics version, with no roadmap for moving forward.

Given Go's folks stance on generics, still a lot of Go code is compilable with gccgo.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#34

Post by pjmlp »

v0y4g3r wrote: post #15
MrBuddyCasino wrote: post #11

So in this case, with some modest performance engineering, Golang is surprisingly fast out of the box, and Rust requires more effort and increasingly less idiomatic code to reach the same result. Is this typical?

I think no. In this particular case it involves how Rust handles lifetimes compared to other languages with garbage collector. Essentially it's a comparison between reference counting and tracing garbage collection based on reacheability.

With the caveat that there are various kinds of "languages with garbage collector", including some where manual resource management is an option.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#35

Post by adrian_b »

netprole wrote: post #17
reacharavindh wrote: post #12

Earlier quoted context omitted.

Exactly, then why does it matter that the author had anything in their post as a figure of speech or analogy? Is it wrong to post a meme of a dog sitting near fire - https://knowyourmeme.com/memes/this-is-fine As a joke from SREs who handle firefighting calls? Does it offend dog lovers, people who are scared of fire?

I don't think I understand the comparison. Can you think of another one that involves a weapon and directed violence?

When I see a knife, it never suggests a weapon, unless it is the kind of knife that is really a weapon, like a double-edged stiletto, or it is wielded by someone who has obvious intentions to use it as a weapon.

The knife from that logo does not look like a weapon, but just like a standard utility knife. Moreover, a crab cannot move a knife in the way in which it is used as a weapon, e.g. for stabbing, but only in a way similar to a human who eats using a knife. Therefore a crab does not suggest someone who uses a knife for violent purposes.

For people like myself, who do not buy industrially-made food, there is no other tool more important than knives. Without using knives every day, I would starve to death.

So you may be offended by seeing a knife that in your mind looks like a weapon, but I am offended when someone claims that one of the most essential, if not the most essential tool of the humans suggests violence or other bad things.

Some people may use knives seldom or never, but then their lives are completely dependent of the work of other humans who use knives to produce the things that sustain the lives of those who do not use knives.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#36

Post by formerly_proven »

sapiogram wrote: post #29
MrBuddyCasino wrote: post #11

So in this case, with some modest performance engineering, Golang is surprisingly fast out of the box, and Rust requires more effort and increasingly less idiomatic code to reach the same result. Is this typical?

I haven't dug into the code, but it's clear that the Golang library has had a ton of optimization work put into it by very knowledgeable people. Techniques like object pooling are highly error prone, and certainly not "out of the box" Golang. The Rust code and the blog post, on the other hand, seem to be written by someone less familiar with Rust and high-performance parsing. I think they would have avoided all their…

I’m unfamiliar with all of the libraries used here, however, the solutions from the blog post struck me as quite odd. serde::Deserializer solves a similar problem and makes it quite straightforward and safe to borrow from the input, if possible. Obviously, the input needs to outlive the borrowed values.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#37

Post by littlestymaar »

sapiogram wrote: post #29
MrBuddyCasino wrote: post #11

So in this case, with some modest performance engineering, Golang is surprisingly fast out of the box, and Rust requires more effort and increasingly less idiomatic code to reach the same result. Is this typical?

I haven't dug into the code, but it's clear that the Golang library has had a ton of optimization work put into it by very knowledgeable people. Techniques like object pooling are highly error prone, and certainly not "out of the box" Golang. The Rust code and the blog post, on the other hand, seem to be written by someone less familiar with Rust and high-performance parsing. I think they would have avoided all their…

> Apart from one "forces one to deal with the contagion of lifetimes" comment in the conclusion

I guess that must be the logical consequence of the “async function are contagious” meme… I wonder if at some point we end up with people arguing that dynamic typing is obviously better because it avoids “contagion with types”.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#38

Post by pigpang »

MrBuddyCasino wrote: post #11

So in this case, with some modest performance engineering, Golang is surprisingly fast out of the box, and Rust requires more effort and increasingly less idiomatic code to reach the same result. Is this typical?

Idiomatic quick-protobuf is much faster than non-idiomatic rust-protobuf: https://github.com/tafia/quick-protobuf/tree/master/perftest

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#39

Post by kjksf »

sapiogram wrote: post #31
riku_iki wrote: post #19

Earlier quoted context omitted.

> but typically rust or c++ would be faster after heavy optimisation; it is concerning how much digging was required to optimize rust code in this case.

The Go code is already hyper-optimized by experts, just not by the blog authors, so you don't read about it here. As someone who has tried to write high-performance Go code on occasion, I can assure you that a ton of digging would have been required on that side as well.

To bring this discussion back to earth, the Go code in question is: https://github.com/VictoriaMetrics/easyproto/blob/master/rea...

I don't see why did you label Aliaksandr Valialkin, the author, an "expert". I mean, he's no dummy but what exactly makes him an expert on optimizing Go code?

As someone who also writes Go, I don't see any "hyper optimizations" in the code. It just decodes the bytes of Protocol Buffer using straightforward code that I would expect a competent developer to write.

It really is just: read bytes from memory ([]byte) and interpret them according to PB spec.

There's only one trick there: unsafeBytesToString() that does no-allocation conversion of []byte to string. This is unsafe in general but safe in their specific case. And I've seen this trick before so it's not some secret, expert-only knowledge.

Most comments here are like bad LLMs: hallucinating opinions without bothering to spent even few minutes acquiring the data to base those opinions on.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#40

Post by pigpang »

fredrikholm wrote: post #28
riku_iki wrote: post #19

Earlier quoted context omitted.

> but typically rust or c++ would be faster after heavy optimisation; it is concerning how much digging was required to optimize rust code in this case.

Whilst still being 1/3rd slower than the Go version.

Rust-protobuf is much slower than quick-protobuf or prost.
Post reply on HN