Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

161–170 of 670 posts

Re: Why Discord is switching from Go to Rust

#161

Post by jolux »

The_rationalist wrote: post #152
pdpi wrote: post #43

Earlier quoted context omitted.

I think the point the GP is trying to make is that there’s no reason why BTreeMap couldn’t be an external crate, while only the core Go collections are allowed to be generic. A corollary to this is that adding more generic collections to Go’s standard library implies expanding the set of magical constructs.

Rust has it's lot of weird hacks too. E.g array can take traits impls only if they have less than 32 elements... https://doc.rust-lang.org/std/array/trait.LengthAtMost32.htm...

That's a completely different and much more minor issue (red herring, more or less) than eschewing the one core language feature that makes performant type-safe custom data structures possible.

Re: Why Discord is switching from Go to Rust

#162

Post by monocasa »

h2odragon wrote: post #37

Excellent write up, and effective argument for Rust in this application and others. My cynical side sums it up as: "Go sucked for us because we refused to own our tooling and make a special allocator for this service. Switching to Rust forced us to do that, and life got better"

They were already not allocating, they were just stuck with a GC cycle that'd scan, not find any garbage, and scan again in two minutes.

Re: Why Discord is switching from Go to Rust

#163

Post by kerkeslager »

biomcgary wrote: post #54
kardianos wrote: post #23

I'm glad they found a good solution (rust) to solve their problem! Also note this was with Go1.9. I know GC work was ongoing during that time, I wonder if this time of situation would still happen?

I know latency for GC with large heaps improved in Go1.12. See: https://golang.org/doc/go1.12#runtime

The article states specifically that part of the problem was heaps were never large.

EDIT: Actually, no it didn't, I misunderstood it.

Re: Why Discord is switching from Go to Rust

#164

Post by zerr »

correct_horse wrote: post #29
RcouF1uZ4gsC wrote: post #25

> Changing to a BTreeMap instead of a HashMap in the LRU cache to optimize memory usage. Collections are one of the big areas where Go's lack of generics really hurts it. In Go, if one of the built in collections does not meet your needs, you are going to take a safety and ergonomic hit going to a custom collection. In Rust, if one of the standard collections does not meet your needs, you (or someone else) can create…

I'm not sure what you mean by standard collections, but BTreeMap is in Rust's standard library.

In Go, standard collections are compiler's magic while in Rust or e.g. C++ - they are implemented as libraries.

Re: Why Discord is switching from Go to Rust

#165

Post by thedance »

The_rationalist wrote: post #159
thedance wrote: post #40

These kinds of posts would be much more interesting if they discussed alternatives considered and rejected. For example why did they choose Rust over C++?

The most pressing undiscussed alternative is: why didn't they update their 3 years old Go version yet had the double standard of using rust nightly... This blog post is a scam and their only reason to use rust should be assumed: it's because they wanted to.

That's exactly the trolling I was looking for :-) . It just reads like they thought it would be cool to use Rust, so they did, which is fine.

Re: Why Discord is switching from Go to Rust

#166

Post by _wldu »

giancarlostoro wrote: post #117
marsokod wrote: post #110

Earlier quoted context omitted.

You can use tabs to indent your python code. Ok, you might be lynched if you share your code but as long as you don't mix tabs and spaces, it is fine.

Same with Go, you can use spaces.

Or none at all: https://play.golang.org/p/VeEaJbJ6sYT

Re: Why Discord is switching from Go to Rust

#167

Post by Uristqwerty »

tasty_freeze wrote: post #135
_ph_ wrote: post #126

Earlier quoted context omitted.

On the good side, the problem of mixing tabs and spaces does normally not appear in Go sources, as gofmt always converts spaces to tabs, so there is no inconsistant indentation. Normally I prefer spaces to tabs because I dislike the mixing, but gofmt solves this nicely for me.

Please explain to me how this works for the case I outlineed, eg: some_function(arg1, arg2, arg3, arg4, arg5, arg6); For the sake of argument, say tabstop=4. If the first line starts with two tabs, will the second line also have two tabs and then a bunch of spaces, or will it start with five tabs and a couple spaces?

In an ideal world, I'd think you would put a "tab stop" character before arg1, then a single tab on the following line, with the bonus benefit that the formatting would survive automatic name changes and not create an indent-change-only line in the diff. Trouble being that all IDEs would have to understand that character, and compilers would have to ignore it (hey, ASCII has form feed and vertical tab that could be repurposed...).

Re: Why Discord is switching from Go to Rust

#168

Post by calcifer »

_ph_ wrote: post #87
flafla2 wrote: post #16

> After digging through the Go source code, we learned that Go will force a garbage collection run every 2 minutes at minimum. In other words, if garbage collection has not run for 2 minutes, regardless of heap growth, go will still force a garbage collection. > We figured we could tune the garbage collector to happen more often in order to prevent large spikes, so we implemented an endpoint on the service to change…

With recent Go releases, GC pauses have become neglible for most applications. So this should not get into your way. However, it can easily tweaked, if needed. There is runtime.ForceGCPeriod, which is a pointer to the forcegcperiod variable. A Go program, which really needs to change this, can do it, but most programs shouldn't require this. Also, it is almost trivial to edit the Go sources (they are included in the…

> especially, as Go is implemented in Go.

Well, parts of it. You can't implement "make" or "new" in Go yourself, for example.

Re: Why Discord is switching from Go to Rust

#169

Post by kerkeslager »

dilyevsky wrote: post #105
kardianos wrote: post #23

I'm glad they found a good solution (rust) to solve their problem! Also note this was with Go1.9. I know GC work was ongoing during that time, I wonder if this time of situation would still happen?

LOL that should’ve been at the top. The improvement in gc between 1.9 and 1.12 is absolutely massive. They could’ve just upgraded go toolchain.

The GC changes in 1.12 supposedly target large heaps, which is not Discord's situation.

Re: Why Discord is switching from Go to Rust

#170

Post by ignoramous »

singron wrote: post #71
ptrincr wrote: post #24

Earlier quoted context omitted.

You are able to disable GC with: GOGC=off As someone mentions below. More details here: https://golang.org/pkg/runtime/

Keeping GC off for a long running service might become problematic. Also, the steady state might have few allocations, but startup may produce a lot of garbage that you might want to evict. I've never done this, but you can also turn GC off at runtime with SetGCPercent(-1). I think with that, you could turn off GC after startup, then turn it back on at desired intervals (e.g. once an hour or after X cache misses). It…

> ...all instances might OOM near the same time.

CloudFront, for this reason, allocates heterogeneous fleets in its PoPs which have diff RAM sizes and CPUs [0], and even different software versions [1].

> When they all restart with cold caches, they might hammer the database again and cause the issue to repeat.

Reminds me of the DynamoDB outage of 2015 that essentially took out us-east-1 [2]. Also, ELB had a similar outage due to unending backlog of work [3].

Someone must write a book on design patterns for distributed system outages or something?

[0] https://youtube.com/watch?v=pq6_Bd24Jsw&t=50m40s

[1] https://youtube.com/watch?v=n8qQGLJeUYAt=39m0s

[2] https://aws.amazon.com/message/5467D2/

[3] https://aws.amazon.com/message/67457/

Post reply on HN