Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

greptime.com

Post reply on HN
Print view View original

21–30 of 63 posts

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#21

Post by netprole »

magicalhippo wrote: post #13
netprole wrote: post #10

Earlier quoted context omitted.

What does a weapon (a knife) have to do with an article about a programming language?

I have no association with any of these communities, but the crab holding a knife was a somewhat well-known meme[1]. I guess it can also be viewed as a play on words, given that crablang is a fork. Given that the creators of crablang explicitly say it was a "lighthearted response"[2] to some of Rust's changes, it makes sense that they'd use a meme for a logo. That said, seems you're not alone in wanting a different l…

Ah, I think I remember that image meme from a long time ago (I never would have connected those dots). Thanks for the context here, it actually helps take the edge off!

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#22

Post by MrBuddyCasino »

v0y4g3r wrote: post #15
MrBuddyCasino wrote: post #11

So in this case, with some modest performance engineering, Golang is surprisingly fast out of the box, and Rust requires more effort and increasingly less idiomatic code to reach the same result. Is this typical?

I think no. In this particular case it involves how Rust handles lifetimes compared to other languages with garbage collector. Essentially it's a comparison between reference counting and tracing garbage collection based on reacheability.

Yes, and this is what I was curious about, because I haven't seen the performance cost of this discussed a lot.

Worded differently: does the strict concept of ownership/lifestimes in Rust bias a default (naive) implementation towards lower performance (eg due to required copying) when compared to a naive Golang (or even Java) implementation?

I have no doubts that after heavy optimization, Rust beats languages such as Go & Java.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#23

Post by magicalhippo »

netprole wrote: post #21
magicalhippo wrote: post #13

Earlier quoted context omitted.

I have no association with any of these communities, but the crab holding a knife was a somewhat well-known meme[1]. I guess it can also be viewed as a play on words, given that crablang is a fork. Given that the creators of crablang explicitly say it was a "lighthearted response"[2] to some of Rust's changes, it makes sense that they'd use a meme for a logo. That said, seems you're not alone in wanting a different l…

Ah, I think I remember that image meme from a long time ago (I never would have connected those dots). Thanks for the context here, it actually helps take the edge off!

That's the challenge with these inside jokes. If you don't get the context, and I didn't recall it instantly either, usually the interpretation will be wildly different.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#24

Post by anonfordays »

netprole wrote: post #2

In the image at the top of the article, why is the Rust crab altered to have "angry" eyes and holding a knife aimed at the Go gopher? Aside from the joke of "don't bring a knife to a protobuf fight" the inference of violence sucks and lessens the spirit of friendly competition and "all in good fun." I don't know if Rust has a code-of-conduct or rules for use of their mascot, but I bet this doesn't follow it.

Congratulations! You've won the Poe's Law Post of the Day Award!

"Poe's law is an adage of Internet culture which says that, without a clear indicator of the author's intent, any parodic or sarcastic expression of extreme views can be mistaken by some readers for a sincere expression of those views."

https://en.m.wikipedia.org/wiki/Poe's_law

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#25

Post by [deleted] »

netprole wrote: post #21
magicalhippo wrote: post #13

Earlier quoted context omitted.

I have no association with any of these communities, but the crab holding a knife was a somewhat well-known meme[1]. I guess it can also be viewed as a play on words, given that crablang is a fork. Given that the creators of crablang explicitly say it was a "lighthearted response"[2] to some of Rust's changes, it makes sense that they'd use a meme for a logo. That said, seems you're not alone in wanting a different l…

Ah, I think I remember that image meme from a long time ago (I never would have connected those dots). Thanks for the context here, it actually helps take the edge off!

[deleted]

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#26

Post by pflanze »

v0y4g3r wrote: post #20
sebstefan wrote: post #18

I got confused through it ; is the resulting code still technically safe?

It's technically `safe` as long as you never access the decoded struct once the original bytes is dropped. I believe that how `unsafe` works, the programmer, instead of compiler, ensures safety.

Yes, but their code appears to actually be `unsafe` (in the Rust terminology sense) without specifying that in their function declarations. They use `unsafe` inside their `slice` function, but return a value that is unsafe to use, hence `slice` should be marked `unsafe`, as should `copy_to_bytes` and then `merge_bytes`. Same for PromLabel::merge_field and PromTimeSeries::merge_field as far as I can see, and maybe higher up in their actual app. This is definitely not how Rust code is supposed to work, if a function isn't marked unsafe, it should not be allowed to introduce UB; they violate that. This approach is on par security wise with C/C++ code iff programmers are aware of the pitfalls, which normally isn't the case since Rust programmers expect non-unsafe functions to be safe (i.e. not require additional care to avoid undefined behaviour).

They either need to mark their functions `unsafe`, or use lifetimes (which may require changes in some APIs, which may be the reason they didn't).

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#27

Post by pflanze »

MrBuddyCasino wrote: post #22
v0y4g3r wrote: post #15

Earlier quoted context omitted.

I think no. In this particular case it involves how Rust handles lifetimes compared to other languages with garbage collector. Essentially it's a comparison between reference counting and tracing garbage collection based on reacheability.

Yes, and this is what I was curious about, because I haven't seen the performance cost of this discussed a lot. Worded differently: does the strict concept of ownership/lifestimes in Rust bias a default (naive) implementation towards lower performance (eg due to required copying) when compared to a naive Golang (or even Java) implementation? I have no doubts that after heavy optimization, Rust beats languages such as…

Using clone or Arc with boxing everywhere to avoid using references with lifetimes at all will lead to code that's slower than Go/Java, yes, unless you're just cloning small objects that don't internally use heap allocations or your algorithm dictates to only need moves, not sharing, or perhaps except it will likely use less RAM which in some situations may still make it faster. But such "newbie" code will probably still be using some other existing code that is using references internally, which makes things faster for those parts. Also, the difficulty of the use of references varies depending on how long / indirect they are going to be used, in many places references are easy to deal with even for a beginner; so it becomes a question of how much the code relies on clone and reference counting.

When I learned Rust, I actually never went with the "use clone or Arc to make your life easier while learning" recommendation but always used references and learned how to use lifetime declarations and program design to go as far with them as reasonable. TBF I had experience with C and C++ already. But once reasonably experienced working in Rust (after a year?), your code should be faster most of the time the way you write it on the first try without needing optimization work.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#28

Post by fredrikholm »

riku_iki wrote: post #19
wokwokwok wrote: post #14

Earlier quoted context omitted.

While it’s a nice bed time story if you like go, the reality is no, it’s not typical. Go has a good perf story, but typically rust or c++ would be faster after heavy optimisation; and should be more or less on par with typical applications. This isn’t a critique of go, and shouldn’t surprise anyone. Typically go also has unexpected optimisation hoops to jump through and problems related to the heavy use of channels (…

> but typically rust or c++ would be faster after heavy optimisation; it is concerning how much digging was required to optimize rust code in this case.

Whilst still being 1/3rd slower than the Go version.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#29

Post by sapiogram »

MrBuddyCasino wrote: post #11

So in this case, with some modest performance engineering, Golang is surprisingly fast out of the box, and Rust requires more effort and increasingly less idiomatic code to reach the same result. Is this typical?

I haven't dug into the code, but it's clear that the Golang library has had a ton of optimization work put into it by very knowledgeable people. Techniques like object pooling are highly error prone, and certainly not "out of the box" Golang.

The Rust code and the blog post, on the other hand, seem to be written by someone less familiar with Rust and high-performance parsing. I think they would have avoided all their problems if they just used lifetimes to safely avoid copying from the start, instead of relying on increasingly elaborate workarounds like the `Bytes` crate. Apart from one "forces one to deal with the contagion of lifetimes" comment in the conclusion, they never mention why they didn't do this, even though it's clearly the idiomatic Rust solution. Maybe they had technical reasons for not doing lifetimes, but to me it just seems like unfamiliarity with Rust.

Re: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance

#30

Post by Filligree »

fredrikholm wrote: post #28
riku_iki wrote: post #19

Earlier quoted context omitted.

> but typically rust or c++ would be faster after heavy optimisation; it is concerning how much digging was required to optimize rust code in this case.

Whilst still being 1/3rd slower than the Go version.

They could have used lifetimes. Instead they used elaborate workarounds, which turned out to be slower.
Post reply on HN