Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

31–40 of 670 posts

Re: Why Discord is switching from Go to Rust

#31

Post by zys5945 »

johnmc408 wrote: post #2

Non programmer here, but would it make sense to add a keyword (or flag) to Go to manually allocate a piece of memory (ie not use GC). That way, for some use cases, you could use avoid GC for the critical path. Then when GC happened, it could be very fast as there would be far less to pause-and-scan (in this use case example). Obviously this would have to be optional and discouraged...but there seems to be no way to w…

That would imply a drastic change to the language design. Essentially you are asking for 2 code generators (one for code managed by the go runtime and one managed by the programmer). It might be possible but it's most likely not gonna happen.

Re: Why Discord is switching from Go to Rust

#32

Post by _bxg1 »

flafla2 wrote: post #16

> After digging through the Go source code, we learned that Go will force a garbage collection run every 2 minutes at minimum. In other words, if garbage collection has not run for 2 minutes, regardless of heap growth, go will still force a garbage collection. > We figured we could tune the garbage collector to happen more often in order to prevent large spikes, so we implemented an endpoint on the service to change…

It does sound like Discord's case was fairly extraordinary in terms of the degree of the spike:

> We kept digging and learned the spikes were huge not because of a massive amount of ready-to-free memory, but because the garbage collector needed to scan the entire LRU cache in order to determine if the memory was truly free from references.

So maybe this is one of those things that just doesn't come up in most cases? Maybe most services also generate enough garbage that that 2-minute maximum doesn't really come into play?

Re: Why Discord is switching from Go to Rust

#34

Post by ericflo »

flafla2 wrote: post #16

> After digging through the Go source code, we learned that Go will force a garbage collection run every 2 minutes at minimum. In other words, if garbage collection has not run for 2 minutes, regardless of heap growth, go will still force a garbage collection. > We figured we could tune the garbage collector to happen more often in order to prevent large spikes, so we implemented an endpoint on the service to change…

It comes from a desire to run in the exact opposite direction as the JVM, which has options for every conceivable parameter. Go has gone through a lot of effort to keep the number of configurable GC parameters to 1.

Re: Why Discord is switching from Go to Rust

#35

Post by afranchuk »

justadudeama wrote: post #18

> Changing to a BTreeMap instead of a HashMap in the LRU cache to optimize memory usage. Can someone explain to me how BTreeMap is more memory efficient than a HashMap?

A BTreeMap should typically have O(n) memory usage, whereas a HashMap (depending on load factor) will usually have O(kn) memory usage, where k > 1. This is because a HashMap allocates the table into which it will store hashed values upfront (and when the load is too great), so it can't anticipate how many values may be added nor what sorts of collisions may occur at this time. Yes, collisions are typically stored as some allocate-per-item collection, but the desire of a HashMap is to avoid such collisions. A BTreeMap allocates for each new value.

Note that this explanation is a bit handwavy, as both data structures have numerous optimizations in production scenarios.

Re: Why Discord is switching from Go to Rust

#36

Post by jhgg »

sascha_sl wrote: post #30
ozten wrote: post #15

Earlier quoted context omitted.

Yes and no. You can get very clever by pre-allocating memory and ensuring it is never garbage collected, but at that point you're opening yourself up to new types of bugs and other performance issues as you try to scale your hack. As you fight your language, you're GC avoidance system will become larger and larger. At some point you might re-evaluate your latency requirements, your architecture, and which are the rig…

Go has a tool for this job. https://golang.org/pkg/sync/#Pool

checked in objects in a sync pool gets cleaned up on GC. It used to clean the whole pool, but now I think it does half each GC cycle. If you want to say "objects checked in should live here forever and not free themselves unless I want them to" sync pool is not the tool for the job.

Re: Why Discord is switching from Go to Rust

#37

Post by h2odragon »

Excellent write up, and effective argument for Rust in this application and others. My cynical side sums it up as:

"Go sucked for us because we refused to own our tooling and make a special allocator for this service. Switching to Rust forced us to do that, and life got better"

Re: Why Discord is switching from Go to Rust

#38

Post by geodel »

correct_horse wrote: post #11
johnmc408 wrote: post #2

Non programmer here, but would it make sense to add a keyword (or flag) to Go to manually allocate a piece of memory (ie not use GC). That way, for some use cases, you could use avoid GC for the critical path. Then when GC happened, it could be very fast as there would be far less to pause-and-scan (in this use case example). Obviously this would have to be optional and discouraged...but there seems to be no way to w…

I think the bigger problem with Go is a lack of GC options. Java, on the other end of the spectrum has multiple GC algorithms (i.e. the Z garbage collector, Shenandoah, Garbage-First/G1) each with tunables (max heap size, min heap size, for more see [1]). Java other issues, but it solves real business problems by having so many garbage collector tunables. Go's philosophy on the matter seems to be that the programmer…

> Java other issues, but it solves real business problems by having so many garbage collector tunables.

That real business problem is Java generates boat load of garbage so GC needs a lot more performance tuning to make application run normal.

Re: Why Discord is switching from Go to Rust

#39

Post by erik_seaberg »

ericflo wrote: post #34
flafla2 wrote: post #16

> After digging through the Go source code, we learned that Go will force a garbage collection run every 2 minutes at minimum. In other words, if garbage collection has not run for 2 minutes, regardless of heap growth, go will still force a garbage collection. > We figured we could tune the garbage collector to happen more often in order to prevent large spikes, so we implemented an endpoint on the service to change…

It comes from a desire to run in the exact opposite direction as the JVM, which has options for every conceivable parameter. Go has gone through a lot of effort to keep the number of configurable GC parameters to 1.

Anyone who pushes the limits of a machine needs tuning options. If you can't turn knobs you have to keep rewriting code until you happen to get the same effect.
Post reply on HN