Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust (2020)

discord.com

Post reply on HN
Print view View original

31–40 of 60 posts

Re: Why Discord is switching from Go to Rust (2020)

#31

Post by mountainriver »

ceeplusplus wrote: post #14
erikbye wrote: post #9

Not discrediting Rust, but I've noticed you rarely hear "we improved performance" by rewriting our implementation using the same language... Although this, too, can yield similar performance improvements.

It does look like with some GC tuning (e.g. manually triggering GC's at a smaller interval than the Go automatic GC threshold) they might've mitigated the spikes, although I don't think they would have gotten the level of perf improvement they did. Golang assembly code IME is not very optimized compared to Rust/C++. edit: reading comprehension skills are lacking, please see comment below for why I'm wrong

They actually would have mitigated it by upgrading their Go version, because the latest release at the time had a fix in the runtime that would have basically solved this.

Turns out no one on the team actually looked into issues in the Go repo to see if it was being addressed. Looks like they just wanted to write Rust, which is fine Rust is cool, but let’s not deceive ourselves.

Re: Why Discord is switching from Go to Rust (2020)

#32

Post by steveklabnik »

mountainriver wrote: post #31
ceeplusplus wrote: post #14

Earlier quoted context omitted.

It does look like with some GC tuning (e.g. manually triggering GC's at a smaller interval than the Go automatic GC threshold) they might've mitigated the spikes, although I don't think they would have gotten the level of perf improvement they did. Golang assembly code IME is not very optimized compared to Rust/C++. edit: reading comprehension skills are lacking, please see comment below for why I'm wrong

They actually would have mitigated it by upgrading their Go version, because the latest release at the time had a fix in the runtime that would have basically solved this. Turns out no one on the team actually looked into issues in the Go repo to see if it was being addressed. Looks like they just wanted to write Rust, which is fine Rust is cool, but let’s not deceive ourselves.

That is not what happened, they did not publish the blog post immediately after the transition, and those changes to the GC did not happen until after the port happened. Some people made assumptions about timeline that were incorrect, and then repeated.

The discussion at the time on Reddit [1] mentions this. The general discussion as well talked about if the improvements, which were big in many cases, would have even improved this particular case. We’ll never truly know.

That said it is important to recognize that Go’s GC has received significant upgrades over the years, and remember that what’s true in the past may not be true today.

1: https://www.reddit.com/r/programming/comments/eyuebc/why_dis...

Re: Why Discord is switching from Go to Rust (2020)

#33

Post by stonewareslord »

erikbye wrote: post #9

Not discrediting Rust, but I've noticed you rarely hear "we improved performance" by rewriting our implementation using the same language... Although this, too, can yield similar performance improvements.

In the article they describe their attempts at tuning and weren't happy with the results

Re: Why Discord is switching from Go to Rust (2020)

#34

Post by robotnaoborot »

bsder wrote: post #27
Xylakant wrote: post #21

Earlier quoted context omitted.

Rust as a language does not protect against memory leaks - std::mem::forget even explicitly does so. Generally garbage collected languages do, so trading go for rust increases the risk of having memory leaks.

You are invoking std::mem::forget which is explicitly for circumventing destructor execution and then complaining about leaks. Okay. The documentation page for std::mem::forget goes through all the alternatives you should try before resorting to std::mem::forget. Now, perhaps std::mem::forget should be marked unsafe. However, you don't just "accidentally" run std::mem::forget. BTW, one of the problems with GC languag…

Depends on gc implementation. With reference counting it's pretty predictable.

Re: Why Discord is switching from Go to Rust (2020)

#35

Post by mohanmcgeek »

jholman wrote: post #17
ceeplusplus wrote: post #14

Earlier quoted context omitted.

It does look like with some GC tuning (e.g. manually triggering GC's at a smaller interval than the Go automatic GC threshold) they might've mitigated the spikes, although I don't think they would have gotten the level of perf improvement they did. Golang assembly code IME is not very optimized compared to Rust/C++. edit: reading comprehension skills are lacking, please see comment below for why I'm wrong

I don't understand.... isn't this idea (triggering GC more often) explicitly discussed in the article?

I remember when this article came out, everybody was pointing out the fact that they used a go version that was several releases older.

Perhaps if the intent wasn't to convince their managers to let them write it in Rust, they would have tried using the latest Go version at the time?

Re: Why Discord is switching from Go to Rust (2020)

#36

Post by mohanmcgeek »

butterisgood wrote: post #23

Well... is this still true? Go's had a lot of perf improvements in the last two years.

It wasn't true even when they wrote the blog. Realistically this should read 2018 because apparently they waited two years before writing this blogpost.

Re: Why Discord is switching from Go to Rust (2020)

#37

Post by afdbcreid »

bsder wrote: post #27
Xylakant wrote: post #21

Earlier quoted context omitted.

Rust as a language does not protect against memory leaks - std::mem::forget even explicitly does so. Generally garbage collected languages do, so trading go for rust increases the risk of having memory leaks.

You are invoking std::mem::forget which is explicitly for circumventing destructor execution and then complaining about leaks. Okay. The documentation page for std::mem::forget goes through all the alternatives you should try before resorting to std::mem::forget. Now, perhaps std::mem::forget should be marked unsafe. However, you don't just "accidentally" run std::mem::forget. BTW, one of the problems with GC languag…

> perhaps std::mem::forget should be marked unsafe

It was in the past IIRC. The problem is that there are multiple ways to leak resources in safe Rust (e.g. creating `Rc` cycles), and the compiler cannot prevent them all. So leaking is safe, and because it's also useful sometimes, `std::mem::forget()` and friends (e.g. `Box::leak()`) are also safe. That being said, it is pretty hard to accidentally leak memory.

Re: Why Discord is switching from Go to Rust (2020)

#38

Post by abeltensor »

jhgg wrote: post #11
phendrenad2 wrote: post #2

(2020) (Anyone know if they're still using Rust?)

Yes we are using rust in a big way. We have multiple teams now full time working on Rust. It is being used on both the client and server, as native modules, web assembly, and also native rust services and NIFs that embed themselves in our elixir services. It has been an incredible success. I plan to blog more about it in the coming months. Our usage of Rust is continuing to grow, and if you check out our jobs page, y…

I love Rust elixir Nifs. Gives you the best of both worlds to be honest. Highly fault tolerant code with fast computation. Only downside is that it can't really handle extreme crashes like a native process can.

Re: Why Discord is switching from Go to Rust (2020)

#39

Post by abeltensor »

erikbye wrote: post #9

Not discrediting Rust, but I've noticed you rarely hear "we improved performance" by rewriting our implementation using the same language... Although this, too, can yield similar performance improvements.

The main issue with Go in this regard is that it generally doesn't have multiple ways of doing the same thing at least idiomatically. I mean sure, there are architectural choices you could make in a rewrite but the actual structure of the code itself is going to be very similar.

Re: Why Discord is switching from Go to Rust (2020)

#40

Post by abeltensor »

jatone wrote: post #20
verdagon wrote: post #18

I suspect that GC'd languages could mitigate this problem by introducing regions; separate areas of memory that cannot point at each other. Pony actors [0] have them, and Cone [1] and Vale [2] are trying new things with them. If golang had this, then it might not ever need to run its GC because it could just fire up a new region for every request. The request will likely end and blast away its memory before it needs…

or you know; just pace the GC mark and sweep algorithm. which is what go is doing now.

You are always going to have some kind of latency spike with a sweeping GC; even if that spike is tiny.
Post reply on HN