Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

21–30 of 670 posts

Re: Why Discord is switching from Go to Rust

#21

Post by JBReefer »

flafla2 wrote: post #16

> After digging through the Go source code, we learned that Go will force a garbage collection run every 2 minutes at minimum. In other words, if garbage collection has not run for 2 minutes, regardless of heap growth, go will still force a garbage collection. > We figured we could tune the garbage collector to happen more often in order to prevent large spikes, so we implemented an endpoint on the service to change…

Go always feels like an amateur language to me, I’ve given up on it. This feels right in line - similar to the hardcode GitHub magic.

Re: Why Discord is switching from Go to Rust

#22

Post by jhgg »

justadudeama wrote: post #18

> Changing to a BTreeMap instead of a HashMap in the LRU cache to optimize memory usage. Can someone explain to me how BTreeMap is more memory efficient than a HashMap?

This is a bit unclear. The root map is still a hash map, but it's a "map of maps" the inner map is a BTreeMap - this is for memory efficiency, as the inner map is relatively smaller and we wouldn't have to deal with the growth factor of a hash map (and having to manually manage that.) where as the root hash map is pre allocated to its max size.

Re: Why Discord is switching from Go to Rust

#24

Post by ptrincr »

flafla2 wrote: post #16

> After digging through the Go source code, we learned that Go will force a garbage collection run every 2 minutes at minimum. In other words, if garbage collection has not run for 2 minutes, regardless of heap growth, go will still force a garbage collection. > We figured we could tune the garbage collector to happen more often in order to prevent large spikes, so we implemented an endpoint on the service to change…

You are able to disable GC with:

  GOGC=off
As someone mentions below.

More details here: https://golang.org/pkg/runtime/

Re: Why Discord is switching from Go to Rust

#25

Post by RcouF1uZ4gsC »

> Changing to a BTreeMap instead of a HashMap in the LRU cache to optimize memory usage.

Collections are one of the big areas where Go's lack of generics really hurts it. In Go, if one of the built in collections does not meet your needs, you are going to take a safety and ergonomic hit going to a custom collection. In Rust, if one of the standard collections does not meet your needs, you (or someone else) can create a pretty much drop-in replacement that does that has similar ergonomic and safety profiles.

Re: Why Discord is switching from Go to Rust

#26

Post by unlinked_dll »

It'd be cool to look at more signal statistics from the CPU plot.

It appears that Go has a lower CPU floor, but it's killed by the GC spikes, presumably due to the large cache mentioned by the author.

This is interesting to me. It suggests that Rust is better at scale than Go, and I would have thought with Go's mature concurrency model and implementation would have been optimized for such cases while Rust would shine in smaller services with CPU bound problems.

Great post!

Re: Why Discord is switching from Go to Rust

#27

Post by jhgg »

mperham wrote: post #20

Better title: "One Discord microservice with extremely high traffic is moving to Rust"

This is one of multiple, we did not blog about this one, but switching a Python http service for analytics ingest that was purely CPU bound to rust resulted in a 90% reduction in compute required to power it. However, that's not too interesting because it's known that Python is slow haha.

We have 2 golang services left, one of them has a rewrite in rust in PR as of last week (as a fun side project an engineer wanted to try out.)

Additionally, as we move towards a more SOA internally, we plan to write more high velocity data services, and rust will be our language of choice for that.

Re: Why Discord is switching from Go to Rust

#29

Post by correct_horse »

RcouF1uZ4gsC wrote: post #25

> Changing to a BTreeMap instead of a HashMap in the LRU cache to optimize memory usage. Collections are one of the big areas where Go's lack of generics really hurts it. In Go, if one of the built in collections does not meet your needs, you are going to take a safety and ergonomic hit going to a custom collection. In Rust, if one of the standard collections does not meet your needs, you (or someone else) can create…

I'm not sure what you mean by standard collections, but BTreeMap is in Rust's standard library.

Re: Why Discord is switching from Go to Rust

#30

Post by sascha_sl »

ozten wrote: post #15
johnmc408 wrote: post #2

Non programmer here, but would it make sense to add a keyword (or flag) to Go to manually allocate a piece of memory (ie not use GC). That way, for some use cases, you could use avoid GC for the critical path. Then when GC happened, it could be very fast as there would be far less to pause-and-scan (in this use case example). Obviously this would have to be optional and discouraged...but there seems to be no way to w…

Yes and no. You can get very clever by pre-allocating memory and ensuring it is never garbage collected, but at that point you're opening yourself up to new types of bugs and other performance issues as you try to scale your hack. As you fight your language, you're GC avoidance system will become larger and larger. At some point you might re-evaluate your latency requirements, your architecture, and which are the rig…

Go has a tool for this job.

https://golang.org/pkg/sync/#Pool

Post reply on HN