Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

241–250 of 670 posts

Re: Why Discord is switching from Go to Rust

#241

Post by kerkeslager »

papaf wrote: post #208
kerkeslager wrote: post #145

Earlier quoted context omitted.

"Simple" when used in programming, doesn't mean anything. So let's be clear here: what we mean is that compilation occurs in a single pass and the artifact of compilation is a single binary. These are two things that make a lot of sense at Google if you read why they were done. But unless you're working at Google, I struggle to guess why you would care about either of these things. The first requires sacrificing anyt…

I don't agree with your definition of simplicity. I like Go and I consider it a simple language because: 1. I can keep most of the language in my head and I don't hit productivity pauses where I have to look something up. 2. There is usually only one way to do things and I don't have to spend time deciding on the right way. For me, these qualities make programming very enjoyable.

> I don't agree with your definition of simplicity.

You mean where I explicitly said that "simple" didn't mean anything, so we should talk about what we mean more concretely?

> 1. I can keep most of the language in my head and I don't hit productivity pauses where I have to look something up.

The core language is currently small, but every language grows with time: even C with its slow-moving, change-averse standards body has grown over the years.

> 2. There is usually only one way to do things and I don't have to spend time deciding on the right way.

Go supports functional programming and object-oriented programming, so pretty much anything you want to do has at least two ways to do it--it sounds like you just aren't familiar with the various ways.

The problem with having more than one way to do things isn't usually choosing which to use, by the way: the problem is when people use one of the many ways differently within the same codebase and it doesn't play nicely with the way things are done in the codebase.

This isn't really a criticism of Go, however: I can't think of a language that actually delivers on there being one right way to do things (most don't even make that promise--Python makes the promise but certainly doesn't deliver on it).

Re: Why Discord is switching from Go to Rust

#242

Post by _ph_ »

jhgg wrote: post #51
bradhe wrote: post #33

Replatforming to solve this problem was a bit silly in my opinion. The solution to the problem was "do fewer allocations" which can be done in any language.

Your reply misses the point. We were already doing so few allocations that the GC only ran because it "had to" at every 2 minute mark. The issue was the large heap of many long lived objects.

Did you try to change that interval to a much larger time?

Re: Why Discord is switching from Go to Rust

#243

Post by kerkeslager »

Corrado wrote: post #201
kerkeslager wrote: post #112

Go is not a general-purpose language. It's a Google language designed to solve Google's problems. If you aren't Google, you probably have different problems, which Go isn't intended to solve. EDIT: Currently at -4 downvotes. Would downvoters care to discuss their votes?

I agree. One of Go's design goals was to be simple enough for thousands of developers to use it simultaneously across a huge monorepo. To me this is in the same class as companies use k8s; unless your Google (or Facebook or Netflix ...) you probably shoudn't be using it.

> To me this is in the same class as companies use k8s; unless your Google (or Facebook or Netflix ...) you probably shoudn't be using it.

I'll actually even say, that if you're Facebook or Netflix, you still shouldn't use Go, because you can write your own tools that solve your problems.

Re: Why Discord is switching from Go to Rust

#244

Post by steveklabnik »

masklinn wrote: post #236
steveklabnik wrote: post #230

Earlier quoted context omitted.

There's no way to hook into the move, so I don't see how it would be possible, or at least, not with techniques similar to compacting GCs.

Maybe by reifying the indirection? The compacting arena would hand out smart pointers which would either always bounce through something (to get from an indentity to the actual memory location, at a cost) or it'd keep track and patch the pointers it handed out somehow . Possibly half and half, I don't remember what language it was (possibly obj-c?) which would hand out pointers, and on needing to move the allocations…

Yeah so I was vaguely wondering about some sort of double indirection; the structure keeps track of "this is a pointer I've handed out", those pointers point into that, which then points into the main structure.

I have no idea if this actually a good idea, seems like you get rid of a lot of the cache locality advantages.

Re: Why Discord is switching from Go to Rust

#245

Post by zorkian »

PeterCorless wrote: post #234
masklinn wrote: post #217

Earlier quoted context omitted.

The good news are that Go's GC has basically no tunables, so you wouldn't have spent weeks on that. The bad news is that it has basically no tunables so if it's a tuning issue you're either fucked or have to put "tuning" hacks right into the code if you find any that works (e.g. twitch's "memory ballast" to avoid overly aggressive GC runs: https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i... )

There are tradeoffs with all languages. C++ avoids the GC, but you then have to make sure you know how to avoid the common pitfalls of that language. We use C++ at Scylla (saw that we got a shout-out in the blog! Woot!) but it's not like there isn't a whole industry about writing blogs avoiding C++ pitfalls. C++ pitfalls through the years... • https://www.horstmann.com/cpp/pitfalls.html (1997) • https://stackoverflow…

(Discord infra person here.)

Actually we didn't update the reference to Cassandra in the article -- the read states workload is now on Scylla too, as of last week. ;)

We'll be writing up a blog post on our migration with Scylla at some point in the next few months, but we've been super happy with it. I replaced our TokuMX cluster with it and it's faster, more reliable, _and_ cheaper (including the support contract). Pretty great for us.

Re: Why Discord is switching from Go to Rust

#246

Post by rob74 »

cies wrote: post #210
deweller wrote: post #179

Earlier quoted context omitted.

As an outsider to both the Go and Rust cultures, I read the Actix news and walked away with the impression that the Rust ecosystem is less mature.

Go's is more pragmatic. Rust's is more purist, and that reflects on the language features (more functional, more free in allowing you to use it for any purpose where Go is network-app specific, more strict in typing), the licensing and the attitude towards collaboration. That collaboration thing is why Actix exploded I think. While mostly an isolated incident it does show some clash between the author's values (and p…

> Go is network-app specific

Just because it is used most for "network apps" doesn't mean it's limited to that. On the other hand, you could argue that Rust is a wrong fit for anything _except_ performance-critical applications, because for anything else it's not worth to saddle yourself with the added complexity.

> and Rust very broad --a true general purpose prog lang-- it is easy to see how Go mature so quickly (not much to mature) and also why it got a bit old so quickly as well

This simplicity is the thing Go opponents like to point out (or mock) most, and what Go fans actually would tell you is one of the best features of the language. It's actually refreshing to have one language that doesn't try to be everyone's darling by implementing every conceivable feature - we already have enough of those, Rust, C++, Java etc. etc. But you don't have to take my word for it, you can also read the first sentences from this blog post: https://bradfitz.com/2020/01/30/joining-tailscale - he puts it better than I could...

Re: Why Discord is switching from Go to Rust

#247

Post by codehalo »

mping wrote: post #171
The_rationalist wrote: post #95

Earlier quoted context omitted.

Out of curiosity, why didn't you choose Kotlin? It can reuse the Java ecosystem which allow you to save tons of money, and give you advanced features and scalability. It is a sexier and more ergonomic language too. And with e.g ZGC, you can have a GC that is fine tunable, and that has very low latency. By choosing rust you will suffer a great deal of the limitations of it's poor, not production ready, ecosystem. I'm…

Ecosystem is a real problem, but motivated engineers will make it work no matter what. I mean, banks run on COBOL. Besides, I am willing to bet idiomatic rust is 2x-10x faster than idiomatic kotlin.

Then that motivation should be applicable to Go as well.

Re: Why Discord is switching from Go to Rust

#248

Post by danbolt »

erk__ wrote: post #222
wongarsu wrote: post #77

Earlier quoted context omitted.

Games written in the Unity engine are (predominately) written in C#, a garbage collected language. Keeping large amounts of data around isn't that unusual since reading from disk is often prohibitively slow, and it's normal to minimize memory allocation/garbage generation (using object pools, caches etc), and manually trigger the GC in loading screens and in other opportune places (as easy as calling System.GC.Collec…

C# uses a generational gc iirc so it may be better suited for a system where you have a relativly stable collection that does not need to be fully garbage collected all the time and have a smaller and more volitile set of objects that will be gc'ed more often. I don't think the current garbage collector in go does anything similar to that.

This might have changed with more recent updates, but I was under the impression that the Mono garbage collector in Unity was a bit dated and not as up-to-date as a C# one today.

Re: Why Discord is switching from Go to Rust

#249

Post by dennisgorelik »

tasty_freeze wrote: post #135
_ph_ wrote: post #126

Earlier quoted context omitted.

On the good side, the problem of mixing tabs and spaces does normally not appear in Go sources, as gofmt always converts spaces to tabs, so there is no inconsistant indentation. Normally I prefer spaces to tabs because I dislike the mixing, but gofmt solves this nicely for me.

Please explain to me how this works for the case I outlineed, eg: some_function(arg1, arg2, arg3, arg4, arg5, arg6); For the sake of argument, say tabstop=4. If the first line starts with two tabs, will the second line also have two tabs and then a bunch of spaces, or will it start with five tabs and a couple spaces?

You should NOT do such alignment anyway, because if you rename "some_function" to "another_function", then you will lose your formatting.

Instead, format arguments in a separate block:

    some_function(
        arg1, arg2, arg3, arg4,
        arg5, arg6);
When arguments are aligned in a separate block, both spaces and tabs work fine.

My own preference is tabs, because of less visual noise in code diff [review].

Re: Why Discord is switching from Go to Rust

#250

Post by masklinn »

steveklabnik wrote: post #244
masklinn wrote: post #236

Earlier quoted context omitted.

Maybe by reifying the indirection? The compacting arena would hand out smart pointers which would either always bounce through something (to get from an indentity to the actual memory location, at a cost) or it'd keep track and patch the pointers it handed out somehow . Possibly half and half, I don't remember what language it was (possibly obj-c?) which would hand out pointers, and on needing to move the allocations…

Yeah so I was vaguely wondering about some sort of double indirection; the structure keeps track of "this is a pointer I've handed out", those pointers point into that, which then points into the main structure. I have no idea if this actually a good idea, seems like you get rid of a lot of the cache locality advantages.

I don't know that the cache locality would be a big issue (your indirection table would be a small-ish array), however you'd eat the cost of doubling the indirections, each pointer access would be two of them.
Post reply on HN