Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

171–180 of 670 posts

Re: Why Discord is switching from Go to Rust

#171

Post by mping »

The_rationalist wrote: post #95
jhgg wrote: post #67

Earlier quoted context omitted.

This blog post perhaps is a bit "after the fact" we had made the switch over mid 2019, and wanted to try out rust as well for services like this, due to adoption elsewhere in the company. Also, after upgrading between 4 golang versions on this service and noticing it didn't materially change performance, we decided to just spend our time on the rewrite (for fun, and latency) and to get a head start into the asynchron…

Out of curiosity, why didn't you choose Kotlin? It can reuse the Java ecosystem which allow you to save tons of money, and give you advanced features and scalability. It is a sexier and more ergonomic language too. And with e.g ZGC, you can have a GC that is fine tunable, and that has very low latency. By choosing rust you will suffer a great deal of the limitations of it's poor, not production ready, ecosystem. I'm…

Ecosystem is a real problem, but motivated engineers will make it work no matter what. I mean, banks run on COBOL.

Besides, I am willing to bet idiomatic rust is 2x-10x faster than idiomatic kotlin.

Re: Why Discord is switching from Go to Rust

#172

Post by geodel »

monocasa wrote: post #121
nickserv wrote: post #76

Earlier quoted context omitted.

It's just an example of something that the Go team took a decision on, and won't allow you to change. I mean, even Python lets you choose. I don't really have a problem with it however, even if I do prefer spaces.

There's a difference between making decisions that are really open to bikeshedding, and making sweeping decisions in contexts that legitimately need per app tuning like immature GCs. The Azul guys get to claim that you don't need to tune their gc, golang doesn't.

Hmm ..this is why Azul's install and configure guide run in hundreds of pages. All the advanced tuning, profiling and configuring OS commands, setting contingency memory pools are perhaps for GCs which Azul does not sell.

Re: Why Discord is switching from Go to Rust

#173

Post by masklinn »

reggieband wrote: post #139

When I see this kind of GC performance, I wonder why you wouldn't change the implementation to use some sort of pool allocator. I am guessing each Read State object is identical to one another (e.g. some kind of struct) so why not pre-allocate your memory budget of objects and just keep an unused list outside of your HasMap? In a way this is even closer to a ring where upon ejection you could write the object to disk…

> When I see this kind of GC performance, I wonder why you wouldn't change the implementation to use some sort of pool allocator.

The allocations were not the issue, the article notes that they did little to no allocations, hence the GC only running on forced triggers (every 2mn)

Re: Why Discord is switching from Go to Rust

#174

Post by The_rationalist »

jhgg wrote: post #67
mangatmodi wrote: post #52

Why would they switch to rust, rather than upgrading from 3 years old version?

This blog post perhaps is a bit "after the fact" we had made the switch over mid 2019, and wanted to try out rust as well for services like this, due to adoption elsewhere in the company. Also, after upgrading between 4 golang versions on this service and noticing it didn't materially change performance, we decided to just spend our time on the rewrite (for fun, and latency) and to get a head start into the asynchron…

Also, after upgrading between 4 golang versions on this service and noticing it didn't materially change performance, we decided to just spend our time on the rewrite So you basically don't read release changelogs of the slow iterating language like go, yet have the double standard of keeping up with rust nightly? Because Go 1.12 explicitly mention performance improvements to it's GC.

You just wanted to do it "for fun" (but is rust and it' s immature ecosystem with all it's issues that fun?). This blog is dishonest and show amateurism at discord.

BTW it's not too late, prove us right or wrong by benchmarcking latest Go GC vs rust.

Re: Why Discord is switching from Go to Rust

#175

Post by Nullabillity »

heinrich5991 wrote: post #107
reificator wrote: post #59

Earlier quoted context omitted.

I could be wrong, but I don't believe there is "hardcoded[d] GitHub magic". IIRC I have used GitLab and Bitbucket and self-hosted Gitea instances the same exact way, and I'm fairly sure there was an hg repo in one of those. Don't recall doing anything out of the ordinary compared to how I would use a github URL.

There are a couple of hosting services hardcoded in Go. I believe it was about splitting the URL into the actual URL and the branch name.

https://github.com/golang/go/blob/e6ebbe0d20fe877b111cf4ccf8...

Ouch, Go never ceases to amaze. The Bitbucket case[0] is even more crazy, calling out to the Bitbucket API to figure out which VCS to use. It has a special case for private repositories, but seems to hard-code cloning over HTTPS.

If only we had some kind of universal way to identify resources, that told you how to access it...

[0]: https://github.com/golang/go/blob/e6ebbe0d20fe877b111cf4ccf8...

Re: Why Discord is switching from Go to Rust

#176

Post by ryuukk_ »

rvcdbn wrote: post #157

Seems like you were hitting: runtime: Large maps cause significant GC pauses #9477 [0] Looks like this issue was resolved for maps that don't contain pointers by [1]. From the article, sounds like the map keys were strings (which do contain pointers, so the map would need to be scanned by the GC). If pointers in the map keys and values could be avoided, it would have (if my understanding is correct) removed the need…

exactly, they use 3 years old version

with the improvements made to runtime and that issue fixed, then is safe to say that GO is much faster than rust, based on their graphs

Re: Why Discord is switching from Go to Rust

#177

Post by cgh »

spullara wrote: post #97
sitkack wrote: post #78

Earlier quoted context omitted.

Systems with poor GC and the need to keep data for lifetimes greater than a request should have an easy to use off heap mechanism to prevent these problems. Often something like Redis is used as a shared cache that is invisible to the garbage collector, there is a natural key with a weak reference (by name) into a KV store. One could embed a KV store into an application that the GC can't scan into.

100%. In Java, you would often use OpenHFT's ChronicleMap for now and hopefully inline classes/records in Java 16 or so.

Ehcache has an efficient off-heap store: https://github.com/Terracotta-OSS/offheap-store/

Doesn't Go have something like this available? It's an obvious thing for garbage-collected languages to have.

Re: Why Discord is switching from Go to Rust

#178

Post by calcifer »

_bxg1 wrote: post #41

It's always good to see a case-study/anecdote, but nothing in here is surprising. It also doesn't really invalidate Go in any way. Rust is faster than Go. People use Go, like any other technology, when the tradeoffs between developer iteration/throughput/latency/etc. make sense. When those cease to make sense, a hot path gets converted down to something more efficient. This is the natural way of things.

This is a weirdly defensive comment, fighting against a strawman. The article doesn't claim it's surprising, that "invalidates" Go or that it isn't the "natural way of things".

Re: Why Discord is switching from Go to Rust

#179

Post by deweller »

chc wrote: post #140
echopom wrote: post #133

This was an extremely interesting read. I'm quiet disappointed though they did not update their Go Version to 1.13[0][1] which would normally have remove the spike issue and thus he latency before they move to Rust... Rust seems more performant with proper usage ( tokio + async ) but I'm more worried about the ecosystem that doesn't seem has mature has Go. We could quote the recent[2] Drama with Actix... [0] https://…

Why would you want to bring up the Actix author's drama? That doesn't seem like something that should reflect on a language one way or the other.

As an outsider to both the Go and Rust cultures, I read the Actix news and walked away with the impression that the Rust ecosystem is less mature.

Re: Why Discord is switching from Go to Rust

#180

Post by anthk »

kick wrote: post #28
correct_horse wrote: post #12

I've heard lots of hot takes on "what Go really is". Here's mine. Go is what would have happened if Bell Labs wrote Java.

Minor nitpick: That already happened, Limbo is what happened when Bell Labs wrote Java.

More like Limbo and Inferno.
Post reply on HN