Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

151–160 of 670 posts

Re: Why Discord is switching from Go to Rust

#151
I wish the article would show a graph of the golang heap usage. I'm reminded of this cloudflare article [0] from a while back where they created an example that seemed to exhibit similar performance issues when they created many small objects to be garbaged collected. They solved it by using a pooled allocator instead of relying solely on the GC. Wonder if that would have been applicable here to the go version.

[0] https://blog.cloudflare.com/recycling-memory-buffers-in-go/

Re: Why Discord is switching from Go to Rust

#152
post #43

Earlier quoted context omitted.

I'm not sure what you mean by standard collections, but BTreeMap is in Rust's standard library.

I think the point the GP is trying to make is that there’s no reason why BTreeMap couldn’t be an external crate, while only the core Go collections are allowed to be generic. A corollary to this is that adding more generic collections to Go’s standard library implies expanding the set of magical constructs.

Rust has it's lot of weird hacks too. E.g array can take traits impls only if they have less than 32 elements... https://doc.rust-lang.org/std/array/trait.LengthAtMost32.htm...

Re: Why Discord is switching from Go to Rust

#154
post #28

I've heard lots of hot takes on "what Go really is". Here's mine. Go is what would have happened if Bell Labs wrote Java.

Minor nitpick: That already happened, Limbo is what happened when Bell Labs wrote Java.

And go is very very derived from plan 9. It could be considered a sibling of limbo in a lot of ways.

Re: Why Discord is switching from Go to Rust

#155
post #126

Earlier quoted context omitted.

On the good side, the problem of mixing tabs and spaces does normally not appear in Go sources, as gofmt always converts spaces to tabs, so there is no inconsistant indentation. Normally I prefer spaces to tabs because I dislike the mixing, but gofmt solves this nicely for me.

Please explain to me how this works for the case I outlineed, eg: some_function(arg1, arg2, arg3, arg4, arg5, arg6); For the sake of argument, say tabstop=4. If the first line starts with two tabs, will the second line also have two tabs and then a bunch of spaces, or will it start with five tabs and a couple spaces?

You wouldn't use an alignment-based style, but a block-based one instead:

  some_function(
      arg1,
      arg2,
      arg3,
      arg4,
      arg5,
      arg6,
  );
(I don't know what Go idiom says here, this is just a more general solution.)

Re: Why Discord is switching from Go to Rust

#156

> Changing to a BTreeMap instead of a HashMap in the LRU cache to optimize memory usage. Can someone explain to me how BTreeMap is more memory efficient than a HashMap?

A BTreeMap should typically have O(n) memory usage, whereas a HashMap (depending on load factor) will usually have O(kn) memory usage, where k > 1. This is because a HashMap allocates the table into which it will store hashed values upfront (and when the load is too great), so it can't anticipate how many values may be added nor what sorts of collisions may occur at this time. Yes, collisions are typically stored as…

> collisions are typically stored as some allocate-per-item collection

Rust's HashMap stores the collisions in the same table as the non-collisions (open addressing), not in a separate collection.

Re: Why Discord is switching from Go to Rust

#157
Seems like you were hitting: runtime: Large maps cause significant GC pauses #9477 [0]

Looks like this issue was resolved for maps that don't contain pointers by [1]. From the article, sounds like the map keys were strings (which do contain pointers, so the map would need to be scanned by the GC).

If pointers in the map keys and values could be avoided, it would have (if my understanding is correct) removed the need for the GC to scan the map. You could do this for example by replacing string keys with fixed size byte arrays. Curious if you experimented this approach?

[0] https://github.com/golang/go/issues/9477

[1] https://go-review.googlesource.com/c/go/+/3288

Re: Why Discord is switching from Go to Rust

#158
post #41

It's always good to see a case-study/anecdote, but nothing in here is surprising. It also doesn't really invalidate Go in any way. Rust is faster than Go. People use Go, like any other technology, when the tradeoffs between developer iteration/throughput/latency/etc. make sense. When those cease to make sense, a hot path gets converted down to something more efficient. This is the natural way of things.

> It's always good to see a case-study/anecdote, but nothing in here is surprising. It also doesn't really invalidate Go in any way.

Well, sure, because categorizing languages as "valid/invalid" doesn't make any sense.

But it does show yet another example of how designing a language to solve Google's fairly-unique problems doesn't result in a general-purpose language suitable for solving most people's problems.

Re: Why Discord is switching from Go to Rust

#159

These kinds of posts would be much more interesting if they discussed alternatives considered and rejected. For example why did they choose Rust over C++?

The most pressing undiscussed alternative is: why didn't they update their 3 years old Go version yet had the double standard of using rust nightly... This blog post is a scam and their only reason to use rust should be assumed: it's because they wanted to.

Re: Why Discord is switching from Go to Rust

#160

> Changing to a BTreeMap instead of a HashMap in the LRU cache to optimize memory usage. Can someone explain to me how BTreeMap is more memory efficient than a HashMap?

A BTreeMap should typically have O(n) memory usage, whereas a HashMap (depending on load factor) will usually have O(kn) memory usage, where k > 1. This is because a HashMap allocates the table into which it will store hashed values upfront (and when the load is too great), so it can't anticipate how many values may be added nor what sorts of collisions may occur at this time. Yes, collisions are typically stored as…

There is no difference between O(n) and O(kn), if k is a constant. The notation deliberately ignores constant factors. (That's why you can say a BTreeMap requires O(n) memory independent of the size or type of data being stored, provided there is some finite upper bound on the sizes of the keys and values.)
Post reply on HN