Why Discord is switching from Go to Rust • Hacker News Board

Re: Why Discord is switching from Go to Rust

#431

Post by marcosdumay » Wed, Feb 05, 2020, 12:52 AM UTC

tormeh wrote: post #416

totalperspectiv wrote: post #399

Earlier quoted context omitted.

Check out this: https://github.com/frol/completely-unscientific-benchmarks

D, Nim and Crystal all do very well on all metrics. Author finds Rust pretty close but not as maintainable. Interesting that the top 3 (performance close to C++, but more maintainable) all are niche languages that haven't really broken into the mainstream.

Rust won't get a maintainability prize for short programs with simple algorithms.

It's Haskell-like features only help on large and complex programs.

Re: Why Discord is switching from Go to Rust

#432

Post by jhgg » Wed, Feb 05, 2020, 12:54 AM UTC

falcolas wrote: post #421

masklinn wrote: post #276

Earlier quoted context omitted.

> While Rust does not have a discrete runtime GC process, it does utilize reference counting for dynamic memory cleanup. That's so misleading as to essentially be a lie. Rust uses reference counting if and only if you opt into it via reference-counted pointers . Using Rc or Arc is not the normal or default course of action, and I'm not aware of any situation where it is ubiquitous. > So you could argue [nonsense] No,…

I was making an assumpotion that using a vector of ARC would be the best way to handle a global LRU cache. Perhaps I should have specified it, but it seemed pretty obvious. Sorry if it wasn’t. If there’s a better way to handle a global LRU cache, I’m all ears.

Assuming only one thread at a time needs to access the LRU cache (not hard with the shared-nothing message passing architecture which we employ here), the lifetime of the object being checked out from the cache is able to be understood at compile time, and we can just use the borrow checker to ensure that it remains that way (we've got a mutable reference to the LRU, and we can use that to get a mutable reference to an object within the LRU. By the time the function that is mutating the data in the LRU finishes, the references to the objects must be dead (the borrow checker will enforce that.) Since all this information is available during compile time, runtime ref-counting (via rc/arc) is not necessary.

This is made possible by rust's memory model, where it understands ownership of data, and the lifetime of each reference that's being taken from that owned data. This means that the compiler can statically determine how long an object needs to live, and that references to the object don't outlive the owned data. For use-cases where the lifetime of references are able to be statically understood, an arc/rc is not required. This blog-post goes into it in much better detail than I can: https://words.steveklabnik.com/borrow-checking-escape-analys...

Re: Why Discord is switching from Go to Rust

#433

Post by CDSlice » Wed, Feb 05, 2020, 1:00 AM UTC

hu3 wrote: post #428

cma wrote: post #410

Earlier quoted context omitted.

He said he felt like a janitor, next guy said he demeaned others as janitors, and now you are saying he demeaned janitors. There is a level of gymnastics going on.

First paragraph: > The common factor in most of my decisions to look for a new job has been realizing that I feel like a very highly compensated janitor instead of a developer. So for that person, feeling like a janitor is incentive for seeking a new job. It's that simple really.

That doesn't mean he is demeaning janitors, just that he doesn't want to be one. There are loads of reasons to not want to be a "code janitor" besides looking down at janitors.

Re: Why Discord is switching from Go to Rust

#434

Post by ascv » Wed, Feb 05, 2020, 1:00 AM UTC

topspin wrote: post #291

ascv wrote: post #111

Earlier quoted context omitted.

It's surprising they didn't test upgrading to 1.13.

> It's surprising they didn't test upgrading to 1.13. It isn't surprising to me. It's stated elsewhere they tried 4 difference version of Go, up through 1.10 apparently, and had performance problems with all of them. At some point you can't suffer garbage collector nonsense anymore and since they'd already employed Rust on other services they tried it here. It worked on the first try. That's not surprising either. Wh…

It's still a huge whoosh. You're starting at 1.9 and you're testing 4 micro versions to 1.10.. what is the point of that? None of those non-major versions are going to significantly change how the GC works.

They could have tried 1 other version (not 4) and picked either the latest (1.13) or the version that contains the GC improvements (1.12) to test. Usually when you are looking to upgrade something you skim the release notes so testing 1.12 or 1.13 is obvious especially when 1.12 seems to specifically address their performance concern.

If upgrading something avoids a service re-write that is usually the way to go unless you were looking for an excuse to re-write the service in the first place which may have been the case.

edit:

It turns out they did exactly what my comment stated: they tested the latest version (1.10). It's just that this article was published recently but the events happened quite a while back.

Re: Why Discord is switching from Go to Rust

#435

Post by arjunbajaj » Wed, Feb 05, 2020, 1:04 AM UTC

robocat wrote: post #394

arjunbajaj wrote: post #356

Question for the Discord team: Was implementing the same service in Elixir an option? Did you try it/why not?

Discord also use Elixir - there are comments elsewhere above for why Elixir might be a bad choice in this case.

Thanks!

Re: Why Discord is switching from Go to Rust

#436

Post by [deleted] » Wed, Feb 05, 2020, 1:09 AM UTC

the-alchemist wrote: post #96

Looks like the big challenge is managing a large, LRU cache, which tends to be a difficult problem for GC runtimes. I bet the JVM, with its myriad tunable GC algorithms, would perform better, especially Shenandoah and, of course, the Azul C4. The JVM world tends to solve this problem by using off-heap caches. See Apache Ignite [0] or Ehcache [1]. I can't speak for how their Rust cache manages memory, but the thing to…

[deleted]

Re: Why Discord is switching from Go to Rust

#437

Post by musicale » Wed, Feb 05, 2020, 1:11 AM UTC

1st Law of Garbage Collection: Consistent speed and efficiency usually requires circumventing the garbage collector.

Re: Why Discord is switching from Go to Rust

#438

Post by brylie » Wed, Feb 05, 2020, 1:15 AM UTC

fatbird wrote: post #368

brylie wrote: post #363

What are some recommended resources for a gentle introduction to Rust?

I read the Rust Programming Language book over Christmas and it's a very good introduction to it, probably one of the best I've seen for any language. It's got a good voice, and it's very good about putting enough context around Rust design decisions to understand the why as well as the how. But's it's not so long that it feels like a slog.

Link, for convenience:

https://doc.rust-lang.org/book/

Re: Why Discord is switching from Go to Rust

#439

Post by corford » Wed, Feb 05, 2020, 1:18 AM UTC

tormeh wrote: post #416

totalperspectiv wrote: post #399

Earlier quoted context omitted.

Check out this: https://github.com/frol/completely-unscientific-benchmarks

D, Nim and Crystal all do very well on all metrics. Author finds Rust pretty close but not as maintainable. Interesting that the top 3 (performance close to C++, but more maintainable) all are niche languages that haven't really broken into the mainstream.

I really wish Intel or MS or someone would fund D so it could give Go and Rust a run for their money. It's as fast (or faster), expressive, powerful and, subjectively, easier to pick up and code in than Rust. It just needs some backers with muscle.

Re: Why Discord is switching from Go to Rust

#440

Post by anarazel » Wed, Feb 05, 2020, 1:19 AM UTC

squarefoot wrote: post #325

georgebarnett wrote: post #303

Earlier quoted context omitted.

Could you please provide a source for this? Java is very fast and 3X slower is a pretty wild claim.

3x might be a bit too much today, but it's definitely slower than C. Also to be considered is the VM overhead, not just the executed code. Here are some benchmarks; I'll leave to the experts out there to confirm or dismiss them. https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Really depends on the domain. There's some things that are a lot easier to make scale up in a language with a great concurrent gc, because that makes writing some lock free data structures quite fundamentally easier (no complicated memory reclamation logic, trivial to avoid ABA problems).