Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

411–420 of 670 posts

Re: Why Discord is switching from Go to Rust

#411

Post by esjeon »

I wonder if they actually did their homework. Doesn't matter if they like it, but they could have avoided rewriting, if they wanted.

The thing is, you can allocate memory outside of Go, and GC will simply ignore such regions, since GC only scan regions known to it. (Mmap should work like a charm here.) A drawback is that pointers in such regions will not be counted, but it's easy to workaround by copying whole data, which is encouraged by the language itself.

TBH, Go sucks for storing a large amount of data. As you can see here, even the simplest cache can be problematic. The language is biased towards large datacenters, where the amount of available resources are less of a concern. Say, this problem can be solved by having external cache servers and extra nodes around them. Latency will not be idealistic, but the service will survive with minimal changes.

Re: Why Discord is switching from Go to Rust

#412

Post by qw »

kaoD wrote: post #188
The_rationalist wrote: post #95

Earlier quoted context omitted.

Out of curiosity, why didn't you choose Kotlin? It can reuse the Java ecosystem which allow you to save tons of money, and give you advanced features and scalability. It is a sexier and more ergonomic language too. And with e.g ZGC, you can have a GC that is fine tunable, and that has very low latency. By choosing rust you will suffer a great deal of the limitations of it's poor, not production ready, ecosystem. I'm…

> you can have a GC But can I not have it?

Yes. You can have a no-op collector

Re: Why Discord is switching from Go to Rust

#413

Post by haimez »

lossolo wrote: post #348
tsimionescu wrote: post #294

Earlier quoted context omitted.

> I can't speak for how their Rust cache manages memory, but the thing to be careful of in non-GC runtimes (especially non-copying GC) is memory fragmentation. As far as I know, a mark-and-sweep collector like Go's doesn't have any advantage over malloc/free when it comes to memory fragmentation. Am I missing some way in which Go's GC helps with fragmentation?

Go GC implementation uses memory allocator that was based on TCMalloc (but derived from it quite a bit). They use a free list of multiple fixed allocatable size-classes, which helps in reducing fragmentation. That's why Go GC is non-copying.

I’m not sure I follow. GC implementations that don’t copy (relocate) are inherently subject to the performance cost of “fragmentation” (in the sense of scattering memory accesses over non-adjacent regions). This is a very high price to pay when you’re dealing with modern hardware.

Re: Why Discord is switching from Go to Rust

#414

Post by cesarb »

faitswulff wrote: post #274
kardianos wrote: post #23

I'm glad they found a good solution (rust) to solve their problem! Also note this was with Go1.9. I know GC work was ongoing during that time, I wonder if this time of situation would still happen?

From /u/DiscordJesse on reddit: > We tried upgrading a few times. 1.8, 1.9, and 1.10. None of it helped. We made this change in May 2019. Just getting around to the blog post now since we've been busy. https://www.reddit.com/r/programming/comments/eyuebc/why_dis...

Another interesting comment in the same reddit thread, from /u/brian-discord (https://old.reddit.com/r/programming/comments/eyuebc/why_dis...):

> Another Discord engineer chiming in here. I worked on trying to fix these spikes on the Go service for a couple weeks. We did indeed try moving up the latest Go at the time (1.10) but this had no effect.

> For a more detailed explanation, it helps to understand what is going on here. It is not the increased CPU utilization that causes the latency. Rather, it's because Go is pausing the entire world for the length of the latency spike. During this time, Go has completely suspended all goroutines which prevents them from doing any work, which appears as latency in requests.

> The specific cause of this seems to be because we used a large free-list like structure, a very long linked list. The head of the list is maintained as a variable, which means that Go's mark phase must start scanning from the head and then pointer chase its way through the list. For whatever reason, Go does (did?) this section in a single-threaded manner with a global lock held. As a result, everything must wait until this extremely long pointer chase occurs.

> It's possible that 1.12 does fix this, but we had tried upgrading a few times already on releases that promised GC fixes and never saw a fix to this issue. I feel the team made a pragmatic choice to divest from Go after giving the language a good attempt at salvaging the project.

Re: Why Discord is switching from Go to Rust

#415

Post by falcolas »

iknowstuff wrote: post #293
falcolas wrote: post #260

Earlier quoted context omitted.

While Rust does not have a discrete runtime GC process, it does utilize reference counting for dynamic memory cleanup. So you could argue that they are still going to suffer some of the downsides of a GC'ed memory allocation. Some potential issues include non-deterministic object lifespan, and ensuring that any unsafe code they write which interacts with the cache does the "right thing" with the reference counts (pot…

I think you're confusing Rust's ownership model with Swift's ARC. Rust doesn't do reference counting unless you use Rc or Arc .

Given the model of memory we are discussing (a global per-process LRU cache), that’s exactly what I was discussing using. Unless there’s another way to handle such global caches.

Re: Why Discord is switching from Go to Rust

#416

Post by tormeh »

totalperspectiv wrote: post #399
bcrosby95 wrote: post #360

Earlier quoted context omitted.

Speaking of, I wish there were an "idiomatic code benchmarks game". Some of us want to compare language speed for common use cases vs trying to squeeze every last piece of performance from it.

Check out this: https://github.com/frol/completely-unscientific-benchmarks

D, Nim and Crystal all do very well on all metrics. Author finds Rust pretty close but not as maintainable. Interesting that the top 3 (performance close to C++, but more maintainable) all are niche languages that haven't really broken into the mainstream.

Re: Why Discord is switching from Go to Rust

#418

Post by correct_horse »

_ph_ wrote: post #212
correct_horse wrote: post #12

I've heard lots of hot takes on "what Go really is". Here's mine. Go is what would have happened if Bell Labs wrote Java.

Interesting comment, as 2 of the main Go creators (Ken Thompson and Rob Pike) did work at the Bell Labs. So while I doubt they tried to write Java, Go in a sense was written by the Bell Labs :). (And Kernighan was their floor-mate too, that must have been a stunningly great environment)

I was taking that into consideration when I authored my comment!

Re: Why Discord is switching from Go to Rust

#420

Post by cmrdporcupine »

takeda wrote: post #318
cmrdporcupine wrote: post #282

Earlier quoted context omitted.

As a Googler, I don't consider this accurate. I've been here 8 years and have yet to work on a Go code base. Yes, there are projects in Go. Certainly not a majority, nor even a significant minority, honestly. No, I wouldn't say Go is specific to Google's problems, though I'm sure some of the engineers had them in mind. I see Go used far more outside of Google than in.

Isn't that indication of a failure? It seems like Go aimed to replace Python and Java code at Google.

My impression (and this was pre-Google and I haven't paid attention since I got here, so) is that it was Rob Pike's and Ken Thompson's project coming out of their long experience with Plan 9 and Inferno/Limbo. That it happened to meet some requirements for some Google projects -- I'm sure that might have been an intent. But that feels a bit like an explanation after the fact, since Go very obviously shows the biases and philosophy from the projects that the original authors had in their previous work.
Post reply on HN