Why Discord is switching from Go to Rust • Hacker News Board

Re: Why Discord is switching from Go to Rust

#321

Post by kerkeslager » Tue, Feb 04, 2020, 10:16 PM UTC

sk0g wrote: post #270

kerkeslager wrote: post #241

Earlier quoted context omitted.

> I don't agree with your definition of simplicity. You mean where I explicitly said that "simple" didn't mean anything, so we should talk about what we mean more concretely? > 1. I can keep most of the language in my head and I don't hit productivity pauses where I have to look something up. The core language is currently small, but every language grows with time: even C with its slow-moving, change-averse standards…

Does Go support functional programming? There's no support for map, filter, etc. It barely supports OOP too, with no real inheritance or generics. I've been happy working with it for a year now, though I've had the chance to work with Kotlin and I have to say, it's very nice too, even if the parallelism isn't quite easy/ convenient to use.

It supports first-class functions, and it supports classes/objects. Sure, it doesn't include good tooling for either, but:

1. map/filter are 2 lines of code each. 2. Inheritance is part of mainstream OOP, but there are some less common languages that don't support inheritance in the way you're probably thinking (i.e. older versions of JavaScript before they caved and introduced two forms of inheritance). 3. Generics are more of a strong type thing than an OOP thing.

Re: Why Discord is switching from Go to Rust

#322

Post by pkolaczk » Tue, Feb 04, 2020, 10:17 PM UTC

gwbas1c wrote: post #223

rvcdbn wrote: post #157

Seems like you were hitting: runtime: Large maps cause significant GC pauses #9477 [0] Looks like this issue was resolved for maps that don't contain pointers by [1]. From the article, sounds like the map keys were strings (which do contain pointers, so the map would need to be scanned by the GC). If pointers in the map keys and values could be avoided, it would have (if my understanding is correct) removed the need…

Everything I've read indicates that RAM caches work poorly in a GC environment. The problem is that garbage collectors are optimized for applications that mostly have short-lived objects, and a small amount of long-lived objects. Things like large in-RAM LRU are basically the slowest thing for a garbage collector to do, because the mark-and-sweep phase always has to go through the entire cache, and because you're con…

A high number of short lived allocations is also a bad thing in a compacting GC environment, because every allocation gets you a reference to a memory region touched very long time ago and it is likely a cache miss. You would like to do an object pool to avoid this but then you run into a pitfall with long living objects, so there is really no good way out.

Re: Why Discord is switching from Go to Rust

#323

Post by marta_morena » Tue, Feb 04, 2020, 10:18 PM UTC

pkolaczk wrote: post #311

This is consistent with my observations of porting Java code to Rust. Much simpler and nicer to read safe Rust code (no unsafe tricks) compiles to programs that outperform carefully tuned Java code.

Sorry, but `Much simpler and nicer` is something that I highly doubt when you talk about Java to Rust. Unless the people writing the Java code were C programmers, lol, in which case I feel for you.

Re: Why Discord is switching from Go to Rust

#324

Post by graphememes » Tue, Feb 04, 2020, 10:20 PM UTC

wvenable wrote: post #279

graphememes wrote: post #141

Earlier quoted context omitted.

Microsoft Word isn't patching the application on startup. That's the difference. Once it's loaded, how much slower than Word is it?

You're telling me Discord is patching itself on every single launch and this somehow a valid excuse for slow startup performance? Almost every single app I run auto-updates itself in some form.

In the case of Discord, yes. That's a valid argument, whether or not it's truly important, I'm not sure. It certainly is a waste of time to invest improving when their current system works perfectly fine.

Re: Why Discord is switching from Go to Rust

#325

Post by squarefoot » Tue, Feb 04, 2020, 10:24 PM UTC

georgebarnett wrote: post #303

novok wrote: post #290

Earlier quoted context omitted.

A C app will tend to outperform a Java or Golang app by 3x, so it isn't too surprising.

Could you please provide a source for this? Java is very fast and 3X slower is a pretty wild claim.

3x might be a bit too much today, but it's definitely slower than C. Also to be considered is the VM overhead, not just the executed code.

Here are some benchmarks; I'll leave to the experts out there to confirm or dismiss them.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Re: Why Discord is switching from Go to Rust

#326

Post by efaref » Tue, Feb 04, 2020, 10:25 PM UTC

marta_morena wrote: post #323

pkolaczk wrote: post #311

This is consistent with my observations of porting Java code to Rust. Much simpler and nicer to read safe Rust code (no unsafe tricks) compiles to programs that outperform carefully tuned Java code.

Sorry, but `Much simpler and nicer` is something that I highly doubt when you talk about Java to Rust. Unless the people writing the Java code were C programmers, lol, in which case I feel for you.

Rust's type system is more expressive than Java's so you can end up with much nicer to read code with stricter and more obvious invariants. There also tends to be way less of the `EnterpriseJavaBeanFactory`-style code in idiomatic Rust.

Re: Why Discord is switching from Go to Rust

#327

Post by sorokod » Tue, Feb 04, 2020, 10:27 PM UTC

gwbas1c wrote: post #223

rvcdbn wrote: post #157

Seems like you were hitting: runtime: Large maps cause significant GC pauses #9477 [0] Looks like this issue was resolved for maps that don't contain pointers by [1]. From the article, sounds like the map keys were strings (which do contain pointers, so the map would need to be scanned by the GC). If pointers in the map keys and values could be avoided, it would have (if my understanding is correct) removed the need…

Everything I've read indicates that RAM caches work poorly in a GC environment. The problem is that garbage collectors are optimized for applications that mostly have short-lived objects, and a small amount of long-lived objects. Things like large in-RAM LRU are basically the slowest thing for a garbage collector to do, because the mark-and-sweep phase always has to go through the entire cache, and because you're con…

In this[1] video at about 32 min, mark there is a discussion on GC and apps that do caching.

[1] https://www.youtube.com/watch?v=VCeHkcwfF9Q

Re: Why Discord is switching from Go to Rust

#328

Post by marcosdumay » Tue, Feb 04, 2020, 10:28 PM UTC

georgebarnett wrote: post #303

novok wrote: post #290

Earlier quoted context omitted.

A C app will tend to outperform a Java or Golang app by 3x, so it isn't too surprising.

Could you please provide a source for this? Java is very fast and 3X slower is a pretty wild claim.

Those people have a really good claim to have the most optimized choice on each language. They've found Java to be 2 to 3 times slower than C and Rust (with much slower outliers).

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

On the real world, you won't get things as optimized in higher level languages, because optimized code looks completely unidiomatic. A 3x speedup from Java is a pretty normal claim.

Re: Why Discord is switching from Go to Rust

#329

Post by mypalmike » Tue, Feb 04, 2020, 10:29 PM UTC

steveklabnik wrote: post #244

masklinn wrote: post #236

Earlier quoted context omitted.

Maybe by reifying the indirection? The compacting arena would hand out smart pointers which would either always bounce through something (to get from an indentity to the actual memory location, at a cost) or it'd keep track and patch the pointers it handed out somehow . Possibly half and half, I don't remember what language it was (possibly obj-c?) which would hand out pointers, and on needing to move the allocations…

Yeah so I was vaguely wondering about some sort of double indirection; the structure keeps track of "this is a pointer I've handed out", those pointers point into that, which then points into the main structure. I have no idea if this actually a good idea, seems like you get rid of a lot of the cache locality advantages.

This sounds a lot like classic MacOS (pre-Darwin) memory allocation. You were allocated a handle, which you called Lock on to get a real pointer. After accessing the memory, you called Unlock to release it. There was definitely a performance hit for that indirection.

Re: Why Discord is switching from Go to Rust

#330

Post by nemothekid » Tue, Feb 04, 2020, 10:29 PM UTC

fjp wrote: post #262

nemothekid wrote: post #232

Earlier quoted context omitted.

The Go community has a very similar story, where someone released a web framework, with an unorthodox set of features, and was flamed to the point where he abandoned the project and quit OSS. https://github.com/go-martini/martini

What was so unorthodox/upsetting to people there?

Martini used the service injection pattern and made use of reflection to do so. It was a very popular framework and one of the first in Go (it currently has ~10k stars), and the use of reflection became a very contentious viewpoint in the community.