I believe the problem described in this blog has been at least partially addressed in the Go 1.12 release.
Why Discord is switching from Go to Rust
521–530 of 670 posts
Re: Why Discord is switching from Go to Rust
#522Earlier quoted context omitted.
Indeed, but it also doesn’t maintain locality of access nearly as well for young objects (the most commonly manipulated ones) and even older ones that survive.
one related point: the article mentions utilizing rust's BTreeMap, which manages its heap allocations with cache efficiency in mind: https://doc.rust-lang.org/std/collections/struct.BTreeMap.ht... . The guts of BTreeMap's memory management code is here: https://github.com/rust-lang/rust/blob/master/src/liballoc/c... . (warning: it is some of the most gnarly rust code I've ever come across, very dense, complex, and he…
I was a bit surprised that BTreeMap turned out to be more memory-efficient than HashMap; can anyone shed some light on this?
One of these days I’ll get around to turning my Rust set implementation[0] into a full-blown map (it’s already [0] https://github.com/senderista/rotated-array-set
Re: Why Discord is switching from Go to Rust
#523Earlier quoted context omitted.
Could you please provide a source for this? Java is very fast and 3X slower is a pretty wild claim.
3x might be a bit too much today, but it's definitely slower than C. Also to be considered is the VM overhead, not just the executed code. Here are some benchmarks; I'll leave to the experts out there to confirm or dismiss them. https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
> 3x might be a bit too much today, but it's definitely slower than C.
If anything the gap is increasing not shrinking. JVM is terrible at memory access patterns due to the design of the language, and designing for memory is increasingly critical for maximum performance on modern systems. All the clever JIT'ing in the world can't save you from the constant pointer chasing, poor cache locality, and poor prefetching.
The gap won't shrink until Java has value types. Which is on the roadmap, yes, but still doesn't exist just yet.
The problem with those benchmarks is if you look at the Java code you'll see it's highly non-idiomatic. Almost no classes or allocations. They almost all exclusively use primitives and raw arrays. Even then it still doesn't match the performance on average of the C (or similar) versions, but if you add the real-world structure you'd find in any substantial project that performance drops off.
Re: Why Discord is switching from Go to Rust
#524Earlier quoted context omitted.
One the one hand, yes. On the other hand, all of this sounds much more complex and fragile. This seems like an important point to me: "Remarkably, we had only put very basic thought into optimization as the Rust version was written. Even with just basic optimization, Rust was able to outperform the hyper hand-tuned Go version."
I found similarly when I ported an image resizing algorithm from Swift to Rust: I'm experienced in swift thus was able to write in an idiomatic way, and have little Rust experience thus I wrote it in a naive way; yet still the rust algorithm was twice(!) as fast. And swift doesn't even have a GC slowing things down!
Reference counting is a (low-throughput, low-latency) form of garbage collection.
Re: Why Discord is switching from Go to Rust
#525Earlier quoted context omitted.
> The JVM world tends to solve this problem by using off-heap caches. See Apache Ignite [0] or Ehcache [1]. For those who care, I was interested how off-heap caching works in Java and I did some quick searching around the Apache Ignite code. The meat is here: - GridUnsafeMemory, an implementation of access to entries allocated off-heap. This appears to implement some common Ignite interface, and invokes calls to a “G…
Unsafe lets you manipulate memory without any JNI overhead other than when allocating or de-allocating memory, and that is usually done in larger chunks and pooled to avoid the overhead at steady state. Netty also takes advantage of Unsafe to move a lot of memory operations off the java heap. Unsafe was one of the cooler aspects to Java that Oracle is actively killing for, well, no good reason at least.
> Unsafe was one of the cooler aspects to Java that Oracle is actively killing for, well, no good reason at least.
I mean, there's the obvious reason that it breaks the memory safety aspect that Java in general guarantees. The whole point of the feature is to subvert the language & expectations.
I'm not saying they should remove it, but it's pretty hard to argue there's "no good reason" to kill it, either. It is, after all, taking the worst parts of C and ramming it into a language that is otherwise immune from that entire class of problems.
Re: Why Discord is switching from Go to Rust
#526Earlier quoted context omitted.
> You have to distinguish between the features available to a Go program as the user writes it and the implementation of the language. I do, I'm just objecting to "Go is implemented in Go".
This reminds me of the ongoing saga of RUSTC_BOOTSTRAP[0][1] The stable compiler is permitted to use unstable features in stable builds, but only for compiling the compiler. In essence, there are some Rust features that are supported by the compiler but only permitted to be used by the compiler. Unsurprisingly, various non-compiler users of Rust have decided that they want those features and begun setting the RUSTC_B…
This is not entirely correct. These things that "can only be used by the compiler" are nightly features that haven't been stabilized yet. Some of them might never be stabilized, but you could always use them in a nightly conpiler, stability assurances just fly out the window then. This is also why using that environment variable is highly discouraged: it breaks the stability guarantees of the language and you're effectively using a pinned nightly. This is reasonable only in a very small handful of cases.
Re: Why Discord is switching from Go to Rust
#527Earlier quoted context omitted.
> I can't speak for how their Rust cache manages memory, but the thing to be careful of in non-GC runtimes (especially non-copying GC) is memory fragmentation. As far as I know, a mark-and-sweep collector like Go's doesn't have any advantage over malloc/free when it comes to memory fragmentation. Am I missing some way in which Go's GC helps with fragmentation?
Go GC implementation uses memory allocator that was based on TCMalloc (but derived from it quite a bit). They use a free list of multiple fixed allocatable size-classes, which helps in reducing fragmentation. That's why Go GC is non-copying.
Sure, but the malloc implementation in Rust probably does something similar.
What I wanted to understand is what is the difference in fragmentation between a non-copying, non-compacting GC and a non-GC runtime.
Re: Why Discord is switching from Go to Rust
#528Earlier quoted context omitted.
Allocator underneath is keeping track of freed memory, so next allocation has high chance of being squeezed into memory region that has been used before. It's obviously not as good as say GC that relocates after sweep, but at least it doesn't leave gaping holes.
Indeed, but it also doesn’t maintain locality of access nearly as well for young objects (the most commonly manipulated ones) and even older ones that survive.
In the current context, fragmentation refers more to the problem of consuming extra memory through fragmentation, which malloc implementations like the one Go (or Rust, or glibc) uses can often mitigate.
Re: Why Discord is switching from Go to Rust
#529Earlier quoted context omitted.
> It's surprising they didn't test upgrading to 1.13. It isn't surprising to me. It's stated elsewhere they tried 4 difference version of Go, up through 1.10 apparently, and had performance problems with all of them. At some point you can't suffer garbage collector nonsense anymore and since they'd already employed Rust on other services they tried it here. It worked on the first try. That's not surprising either. Wh…
It's still a huge whoosh. You're starting at 1.9 and you're testing 4 micro versions to 1.10.. what is the point of that? None of those non-major versions are going to significantly change how the GC works. They could have tried 1 other version (not 4) and picked either the latest (1.13) or the version that contains the GC improvements (1.12) to test. Usually when you are looking to upgrade something you skim the rel…
According to another comment they did this back in May 2019 when 1.10 was the latest. They are only blogging about it now which I guess is slightly unfortunate but never the less.
Re: Why Discord is switching from Go to Rust
#530Earlier quoted context omitted.
I found similarly when I ported an image resizing algorithm from Swift to Rust: I'm experienced in swift thus was able to write in an idiomatic way, and have little Rust experience thus I wrote it in a naive way; yet still the rust algorithm was twice(!) as fast. And swift doesn't even have a GC slowing things down!
> Swift doesn't have a GC slowing things down This Apple marketing meme needs to die. Reference counting incurs arguably more cost than GC, or at least the cost is spread through-out processing.
It incurs some cost, but whether it is higher is very debatable. This is very much workload dependent. A smart compiler can elide most reference updates.