Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

261–270 of 670 posts

Re: Why Discord is switching from Go to Rust

#261

Post by typical182 »

jhgg wrote: post #67
mangatmodi wrote: post #52

Why would they switch to rust, rather than upgrading from 3 years old version?

This blog post perhaps is a bit "after the fact" we had made the switch over mid 2019, and wanted to try out rust as well for services like this, due to adoption elsewhere in the company. Also, after upgrading between 4 golang versions on this service and noticing it didn't materially change performance, we decided to just spend our time on the rewrite (for fun, and latency) and to get a head start into the asynchron…

Do you have any load tests or synthetic benchmarks that are still capable of producing this?

It would be interesting to see what a more modern Go would do given there have been a bunch of tail latency GC improvements since your older 1.9 Go version... and in an ideal world, it would be nice to file an issue on the tracker if you were still seeing this.

(Maybe that ends up later helping another one of your Go services, or maybe it just helps the community, or maybe it’s a topic for another interesting blog...).

In any event, thanks for taking the time to write up and share this one.

Re: Why Discord is switching from Go to Rust

#262

Post by fjp »

nemothekid wrote: post #232
cies wrote: post #210

Earlier quoted context omitted.

Go's is more pragmatic. Rust's is more purist, and that reflects on the language features (more functional, more free in allowing you to use it for any purpose where Go is network-app specific, more strict in typing), the licensing and the attitude towards collaboration. That collaboration thing is why Actix exploded I think. While mostly an isolated incident it does show some clash between the author's values (and p…

The Go community has a very similar story, where someone released a web framework, with an unorthodox set of features, and was flamed to the point where he abandoned the project and quit OSS. https://github.com/go-martini/martini

What was so unorthodox/upsetting to people there?

Re: Why Discord is switching from Go to Rust

#264

Post by _ph_ »

robocat wrote: post #253
_ph_ wrote: post #87

Earlier quoted context omitted.

With recent Go releases, GC pauses have become neglible for most applications. So this should not get into your way. However, it can easily tweaked, if needed. There is runtime.ForceGCPeriod, which is a pointer to the forcegcperiod variable. A Go program, which really needs to change this, can do it, but most programs shouldn't require this. Also, it is almost trivial to edit the Go sources (they are included in the…

You still may have significant CPU overhead from the GC e.g. the twitch article (mentioned elsewhere in comments) measured 30% CPU used for GC for one program (Go 1.5 I think). Obviously they consider spending 50% more on hardware is a worthwhile compromise for the gains they get (e.g. reduction of developer hours and reduced risk of security flaws or avoiding other effects of invalid pointers).

In this case, as they were running into the automatic GC interval, their program did not create much, if any garbage. So the CPU overhead for the GC would have been quite small.

If you do a lot of allocations, the GC overhead rises of course, but also would the effort of doing allocations/deallocations with a manual managing scheme. In the end it is a bit trade-off, what fits the problem at hand best. The nice thing about Rust is, that "manual" memory management doesn't come at the price of program correctness.

Re: Why Discord is switching from Go to Rust

#265

Post by andoriyu »

_ph_ wrote: post #87
flafla2 wrote: post #16

> After digging through the Go source code, we learned that Go will force a garbage collection run every 2 minutes at minimum. In other words, if garbage collection has not run for 2 minutes, regardless of heap growth, go will still force a garbage collection. > We figured we could tune the garbage collector to happen more often in order to prevent large spikes, so we implemented an endpoint on the service to change…

With recent Go releases, GC pauses have become neglible for most applications. So this should not get into your way. However, it can easily tweaked, if needed. There is runtime.ForceGCPeriod, which is a pointer to the forcegcperiod variable. A Go program, which really needs to change this, can do it, but most programs shouldn't require this. Also, it is almost trivial to edit the Go sources (they are included in the…

> Also, it is almost trivial to edit the Go sources (they are included in the distribution) and rebuild it, which usually takes just a minute. So Go is really suited for your own experiments - especially, as Go is implemented in Go.

Ruby 1.8.x wants to say "Hello"

Re: Why Discord is switching from Go to Rust

#266

Post by ncmncm »

therockhead wrote: post #127
The_rationalist wrote: post #95

Earlier quoted context omitted.

Out of curiosity, why didn't you choose Kotlin? It can reuse the Java ecosystem which allow you to save tons of money, and give you advanced features and scalability. It is a sexier and more ergonomic language too. And with e.g ZGC, you can have a GC that is fine tunable, and that has very low latency. By choosing rust you will suffer a great deal of the limitations of it's poor, not production ready, ecosystem. I'm…

> By choosing rust you will suffer a great deal of the limitations of it's poor, not production ready, ecosystem. Why do you think that? Seems like Rust is a great choice for this type of high performance work.

Rust does best when the number of lines of code that must be parsed in an edit-compile-test loop is small. When the sources that must be parsed get large, coders suffer.

It is doubtful that this will improve, much, without breaking changes to the language. The range of code over which type inference operates, or at least programmers' reliance on it, would need to contract by quite a lot. There would be Complaints.

Re: Why Discord is switching from Go to Rust

#267

Post by PeterCorless »

zorkian wrote: post #245
PeterCorless wrote: post #234

Earlier quoted context omitted.

There are tradeoffs with all languages. C++ avoids the GC, but you then have to make sure you know how to avoid the common pitfalls of that language. We use C++ at Scylla (saw that we got a shout-out in the blog! Woot!) but it's not like there isn't a whole industry about writing blogs avoiding C++ pitfalls. C++ pitfalls through the years... • https://www.horstmann.com/cpp/pitfalls.html (1997) • https://stackoverflow…

(Discord infra person here.) Actually we didn't update the reference to Cassandra in the article -- the read states workload is now on Scylla too, as of last week. ;) We'll be writing up a blog post on our migration with Scylla at some point in the next few months, but we've been super happy with it. I replaced our TokuMX cluster with it and it's faster, more reliable, _and_ cheaper (including the support contract).…

Woot! Go you! (Or Rust you! Whichever you prefer!)

Re: Why Discord is switching from Go to Rust

#268

Post by runevault »

azakai wrote: post #227
monocasa wrote: post #144

Earlier quoted context omitted.

Yeah, they already weren't allocating, it was a GC pause that just scanned and would come up with essentially no extra garbage every two minutes.

A pool allocator could have reduced the number of existing allocations (1 big one instead of many small ones), making those spikes less significant. (But that depends on how Go handles interior pointers and GC, so I'm not sure.)

Allocations weren't the problem. It was the fact that, every 2 minutes, the GC would trigger because of an arbitrary decision by the Go team and scan their entire heap, find little to nothing to deallocate, then go on its merry way.

Re: Why Discord is switching from Go to Rust

#270

Post by sk0g »

kerkeslager wrote: post #241
papaf wrote: post #208

Earlier quoted context omitted.

I don't agree with your definition of simplicity. I like Go and I consider it a simple language because: 1. I can keep most of the language in my head and I don't hit productivity pauses where I have to look something up. 2. There is usually only one way to do things and I don't have to spend time deciding on the right way. For me, these qualities make programming very enjoyable.

> I don't agree with your definition of simplicity. You mean where I explicitly said that "simple" didn't mean anything, so we should talk about what we mean more concretely? > 1. I can keep most of the language in my head and I don't hit productivity pauses where I have to look something up. The core language is currently small, but every language grows with time: even C with its slow-moving, change-averse standards…

Does Go support functional programming? There's no support for map, filter, etc. It barely supports OOP too, with no real inheritance or generics.

I've been happy working with it for a year now, though I've had the chance to work with Kotlin and I have to say, it's very nice too, even if the parallelism isn't quite easy/ convenient to use.

Post reply on HN