Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

181–190 of 670 posts

Re: Why Discord is switching from Go to Rust

#181

Post by steveklabnik »

The_rationalist wrote: post #152
pdpi wrote: post #43

Earlier quoted context omitted.

I think the point the GP is trying to make is that there’s no reason why BTreeMap couldn’t be an external crate, while only the core Go collections are allowed to be generic. A corollary to this is that adding more generic collections to Go’s standard library implies expanding the set of magical constructs.

Rust has it's lot of weird hacks too. E.g array can take traits impls only if they have less than 32 elements... https://doc.rust-lang.org/std/array/trait.LengthAtMost32.htm...

This is purely temporary; it used to be less hacky, but in order to move to the no-hacks world, we had to make it a bit more hacky to start.

Re: Why Discord is switching from Go to Rust

#182

Post by monocasa »

geodel wrote: post #172
monocasa wrote: post #121

Earlier quoted context omitted.

There's a difference between making decisions that are really open to bikeshedding, and making sweeping decisions in contexts that legitimately need per app tuning like immature GCs. The Azul guys get to claim that you don't need to tune their gc, golang doesn't.

Hmm ..this is why Azul's install and configure guide run in hundreds of pages. All the advanced tuning, profiling and configuring OS commands, setting contingency memory pools are perhaps for GCs which Azul does not sell.

I mean, they'll let you because the kind of customers want to be able to are the kinds of customers that Azul targets. But everything I've heard from their engineers is that they've solved a lot of customer problems by resetting things to defaults and just letting it have a giant heap to play with.

Not sure how that makes the golang position any better.

Re: Why Discord is switching from Go to Rust

#183

Post by kerkeslager »

nemo1618 wrote: post #93

I wonder if it would be feasible to rewrite the LRU cache (either fully or in part) in a way that does not require the GC to scan the entire cache.

Yes, it's possible: that's generational garbage collection. But last I heard, Google decided writing a modern GC was too complicated.

They're probably right, because Google doesn't need it. But for everyone else who decided to use a language designed to solve Google's fairly-unique problems as if it were a general-purpose language: that kind of sucks, doesn't it?

Re: Why Discord is switching from Go to Rust

#184

Post by cellularmitosis »

jhgg wrote: post #134
sorokod wrote: post #46

Earlier quoted context omitted.

Running every two minutes sounds pretty deterministic.

It was perhaps too deterministic. What's not mentioned in the blog is that after running for long enough, the cluster would line up it's GCs, and each node would do the 2 minute GC at exactly the same time causing bigger spikes as the entire cluster would degrade. I'm guessing all it takes is a few day night cycles combined with a spike in traffic to make all the nodes reset their forced GC timers to the same time.

Interesting, sounds like 2 minutes + random fuzz might avoid the thundering herd. Might be worth submitting a patch to the golang team!

Re: Why Discord is switching from Go to Rust

#185

Post by afranchuk »

nybble41 wrote: post #160
afranchuk wrote: post #35

Earlier quoted context omitted.

A BTreeMap should typically have O(n) memory usage, whereas a HashMap (depending on load factor) will usually have O(kn) memory usage, where k > 1. This is because a HashMap allocates the table into which it will store hashed values upfront (and when the load is too great), so it can't anticipate how many values may be added nor what sorts of collisions may occur at this time. Yes, collisions are typically stored as…

There is no difference between O(n) and O(kn), if k is a constant. The notation deliberately ignores constant factors. (That's why you can say a BTreeMap requires O(n) memory independent of the size or type of data being stored, provided there is some finite upper bound on the sizes of the keys and values.)

Yeah I know, it was just the fastest way to indicate that the constant factor was almost definitely larger for HashMaps. But thank you for clarifying!

Re: Why Discord is switching from Go to Rust

#186

Post by Communitivity »

chc wrote: post #140
echopom wrote: post #133

This was an extremely interesting read. I'm quiet disappointed though they did not update their Go Version to 1.13[0][1] which would normally have remove the spike issue and thus he latency before they move to Rust... Rust seems more performant with proper usage ( tokio + async ) but I'm more worried about the ecosystem that doesn't seem has mature has Go. We could quote the recent[2] Drama with Actix... [0] https://…

Why would you want to bring up the Actix author's drama? That doesn't seem like something that should reflect on a language one way or the other.

Agreed. One could argue that a level of drama in the community is a sign of growing maturity and wider interest in the language, because it is evidence there is no longer a niche monoculture of devs all thinking the same way.

In the words of Steve Klabnik "Rust has been an experiment in community building as much as an experiment in language building. Can we reject the idea of a BDFL? Can we include as many people as possible? Can we be welcoming to folks who historically have not had great representation in open source? Can we reject contempt culture? Can we be inclusive of beginners?" https://words.steveklabnik.com/a-sad-day-for-rust

The Actix issue was resolved, and Actix will continue under new maintainers (https://github.com/actix/actix-web/issues/1289). So I'd argue the answer to those questions is a 'yes'.

Re: Why Discord is switching from Go to Rust

#187

Post by masklinn »

The_rationalist wrote: post #152
pdpi wrote: post #43

Earlier quoted context omitted.

I think the point the GP is trying to make is that there’s no reason why BTreeMap couldn’t be an external crate, while only the core Go collections are allowed to be generic. A corollary to this is that adding more generic collections to Go’s standard library implies expanding the set of magical constructs.

Rust has it's lot of weird hacks too. E.g array can take traits impls only if they have less than 32 elements... https://doc.rust-lang.org/std/array/trait.LengthAtMost32.htm...

That's… not that at all. You can absolutely implement traits for arrays of more than 32 elements[0].

It is rather that due to a lack of genericity (namely const generics) you can't implement traits for [T;N], you have to implement them for each size individually. So there has to be an upper bound somehow[1], and the stdlib developers arbitrarily picked 32 for stdlib traits on arrays.

A not entirely dissimilar limit tends to be placed on tuples, and implementing traits / typeclasses / interfaces on them. Again the stdlib has picked an arbitrary limit, here 12[2], the same issue can be seen in e.g. Haskell (where Show is "only" instanced on tuples up to size 15).

These are not "weird hacks", they're logical consequences of memory and file size not being infinite, so if you can't express something fully generically… you have to stop at one point.

[0] here's 47 https://play.rust-lang.org/?version=stable&mode=debug&editio...

[1] even if you use macros to codegen your impl block

[2] https://doc.rust-lang.org/src/core/fmt/mod.rs.html#2115

Re: Why Discord is switching from Go to Rust

#188

Post by kaoD »

The_rationalist wrote: post #95
jhgg wrote: post #67

Earlier quoted context omitted.

This blog post perhaps is a bit "after the fact" we had made the switch over mid 2019, and wanted to try out rust as well for services like this, due to adoption elsewhere in the company. Also, after upgrading between 4 golang versions on this service and noticing it didn't materially change performance, we decided to just spend our time on the rewrite (for fun, and latency) and to get a head start into the asynchron…

Out of curiosity, why didn't you choose Kotlin? It can reuse the Java ecosystem which allow you to save tons of money, and give you advanced features and scalability. It is a sexier and more ergonomic language too. And with e.g ZGC, you can have a GC that is fine tunable, and that has very low latency. By choosing rust you will suffer a great deal of the limitations of it's poor, not production ready, ecosystem. I'm…

> you can have a GC

But can I not have it?

Re: Why Discord is switching from Go to Rust

#189

Post by Hamuko »

jrockway wrote: post #100
geodel wrote: post #74

Makes sense write most efficient stuff for in-house and give resource hog Electron apps to users.

Discord pays for their servers, but not for their users's computers.

That's fine as long as you ignore the fact that the users are the customers.
Post reply on HN