Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

651–660 of 670 posts

Re: Why Discord is switching from Go to Rust

#651

Post by ernst_klim »

pkolaczk wrote: post #634
ernst_klim wrote: post #585

Earlier quoted context omitted.

> but it's generally better than most full GC solutions I doubt that. It implies huge costs without giving any benefits of GC. A typical GC have compaction, nearly stack-like fast allocation [1], ability to allocate a bunch of objects at once (just bump the heap pointer once for a whole bunch). And both Perl and Swift do indeed perform abysmally, usually worse than both GC and manual languages [2]. > ARC is more of l…

It is nowhere near stack-like. Stack is hot in cache. Heap memory in tlab is cold. Bringing the lines into cache is the major cost, not bumping the pointer.

> Stack is hot in cache. Heap memory in tlab is cold.

What? This doesn't make any sense. From the cache's POV stack and bump-allocated heap are the same thing. Both are continuous chunks of memory where the next value is being allocated right after the previous one.

The only difference between the stack and the bump-allocated heap is that the former has hardware support for pointer bumping and the latter has not. That's all.

Re: Why Discord is switching from Go to Rust

#652

Post by pkolaczk »

ernst_klim wrote: post #651
pkolaczk wrote: post #634

Earlier quoted context omitted.

It is nowhere near stack-like. Stack is hot in cache. Heap memory in tlab is cold. Bringing the lines into cache is the major cost, not bumping the pointer.

> Stack is hot in cache. Heap memory in tlab is cold. What? This doesn't make any sense. From the cache's POV stack and bump-allocated heap are the same thing. Both are continuous chunks of memory where the next value is being allocated right after the previous one. The only difference between the stack and the bump-allocated heap is that the former has hardware support for pointer bumping and the latter has not. Tha…

You're missing the fact that the tlab pointer is only ever moved forward, so it always points to recently unused memory. Until the reset happens and it points back to the same memory again, the application managed to allocate several megabytes or sometimes hundreds of megabytes, and most of that new-gen memory does not fit even in L3 cache.

The stack pointer moves both directions and the total range of that back-and-forth movement is typically in kilobytes, so it may fit fully in L1.

Just check with perf what happens when you iterate over an array of 100 MB several times and compare that to iterating over 10 kB several times. Both are contiguous but the performance difference is pretty dramatic.

Besides that, there is also an effect that the faster you allocate, the faster you run out of new gen space, and the faster you trigger minor collections. These are not free. The faster you do minor collections, the more likely it is for the objects to survive. And the cost is proportional to survival rate. That's why many Java apps tend to use pretty big new generation size, hoping that before collection happens, most of young objects die.

This is not just theory - I saw this just too many times, when reducing allocation rate to nearly zero caused significant speedups - by order of magnitude of more. Reducing memory traffic is also essential to get good multicore scaling. It doesn't matter each core has a separate tlab, when their total allocation rate is so high that they are saturating LLC - main memory link. It is easy to miss this problem by classic method profiling, because a program with such problem will manifest by just everything being magically slow, but no obvious bottleneck.

Re: Why Discord is switching from Go to Rust

#653

Post by pkolaczk »

BenFrantzDale wrote: post #646
pkolaczk wrote: post #547

Earlier quoted context omitted.

GC makes it easier to write, but not necessarily better. Modern state-of-the-art Java GCs operate a global heap, so you often pay for what you do not use. In languages like Rust or C++ your can build small locally GCed heaps, where you can limit GC overhead to just a few particular structures that need GC, not everything. Therefore other structures like caches don't affect GC pause times. And the "hardness" of writin…

“GC makes it easier to write.” I disagree. GC means I don’t get deterministic destruction which means I can’t use RAII properly.

The post was about writing lockless structures. With lockless RAII is of no use. See ABA problem. RAII is awesome in other places.

Re: Why Discord is switching from Go to Rust

#654

Post by yazaddaruvala »

Thaxll wrote: post #606
yazaddaruvala wrote: post #516

Earlier quoted context omitted.

The graphs were in different units. The final Rust version was over 100x faster.

Which doesn't make any sense. Rust is not x100 faster than Go.

Rust and Go likely translate into similar enough assembly for similar code to make the performance close enough.

However, bigger caches will always have more cache hits than smaller caches. Therefore could easily be 100x faster.

The blog does a better job explaining everything than I can but simply put the “granular” memory management Rust allows gave them an improved ability to create a bigger cache. Go (at the time) while great did not work well for that particular usecase and required smaller cache sizes.

Re: Why Discord is switching from Go to Rust

#655

Post by ernst_klim »

pkolaczk wrote: post #652
ernst_klim wrote: post #651

Earlier quoted context omitted.

> Stack is hot in cache. Heap memory in tlab is cold. What? This doesn't make any sense. From the cache's POV stack and bump-allocated heap are the same thing. Both are continuous chunks of memory where the next value is being allocated right after the previous one. The only difference between the stack and the bump-allocated heap is that the former has hardware support for pointer bumping and the latter has not. Tha…

You're missing the fact that the tlab pointer is only ever moved forward, so it always points to recently unused memory. Until the reset happens and it points back to the same memory again, the application managed to allocate several megabytes or sometimes hundreds of megabytes, and most of that new-gen memory does not fit even in L3 cache. The stack pointer moves both directions and the total range of that back-and-…

> You're missing the fact that the tlab pointer is only ever moved forward, so it always points to recently unused memory. Until the reset happens and it points back to the same memory again, the application managed to allocate several megabytes or sometimes hundreds of megabytes, and most of that new-gen memory does not fit even in L3 cache.

Yes, you are right about stack locality. It indeed moves back and forward making effective used memory region quite small.

> These are not free. The faster you do minor collections, the more likely it is for the objects to survive. And the cost is proportional to survival rate.

Yes, that's true. Immutable languages are doing way better here having small minor heaps (OCaml has 2MB on amd64) and very small survival rates (with many object being directly allocated on older heap if they are known to be lasting in advance).

Now I understand your point better and I agree.

Re: Why Discord is switching from Go to Rust

#656

Post by _ph_ »

giornogiovanna wrote: post #524
chubs wrote: post #307

Earlier quoted context omitted.

I found similarly when I ported an image resizing algorithm from Swift to Rust: I'm experienced in swift thus was able to write in an idiomatic way, and have little Rust experience thus I wrote it in a naive way; yet still the rust algorithm was twice(!) as fast. And swift doesn't even have a GC slowing things down!

Reference counting is a (low-throughput, low-latency) form of garbage collection.

The low-latency part might not even be true. RC means that you don't have CPU consuming heap scans, but if you free the last reference to a large tree of objects, freeing them can take quite a lot of time, causing high latencies.

Re: Why Discord is switching from Go to Rust

#657

Post by _ph_ »

novok wrote: post #290
dochtman wrote: post #256

Earlier quoted context omitted.

One the one hand, yes. On the other hand, all of this sounds much more complex and fragile. This seems like an important point to me: "Remarkably, we had only put very basic thought into optimization as the Rust version was written. Even with just basic optimization, Rust was able to outperform the hyper hand-tuned Go version."

A C app will tend to outperform a Java or Golang app by 3x, so it isn't too surprising.

This statement is definitely wrong in this generic blank fashion. Also I would lay upon you the burden of proof for it :).

Tight, low level code in Java and Go is roughly as fast as average C code. The Go compiler is know to be less good at optimizing code than e.g. GCC, but this in many cases creates little practical difference, while the Java JIT compilers have become excellent to a point where they often beat GCC, especially as they can use run time profiling for code optimization. So they can optimize the code for the actual task at hand.

Where the languages differ in "speed" is their runtime environment. Java and Go are languages with garbage collection, which of course means that some amount of CPU is required to perform GC. But as the modern garbage collectors run in parallel with the program, this CPU effort often enough is no bottleneck. On the other side, manual memory management has different performance trade-offs, which in many cases can make it quite slow on its own.

Re: Why Discord is switching from Go to Rust

#658

Post by wool_gather »

kllrnohj wrote: post #627
wool_gather wrote: post #624

Earlier quoted context omitted.

When you say "value types" in this context, you mean "copy-only types"?

I mean "value types" https://www.jesperdj.com/2015/10/04/project-valhalla-value-t... & http://cr.openjdk.java.net/~jrose/values/values-0.html I don't know what you mean by "copy-only types." I'm not finding any reference to that terminology in the context of language design.

Ah, thanks for the link; I wasn't sure what it meant in the context of Java, since it's possible to get value semantics using a class.

Sorry about the confusion. I meant for the quotes around "copy-only" to indicate that it wasn't really a standard term, but I marked "value types" the same way, so that didn't really work. By "copy-only" I meant something you couldn't have more than one reference to: every name (variable) to which you assign the data would have its own independent copy.

Re: Why Discord is switching from Go to Rust

#659

Post by kllrnohj »

wool_gather wrote: post #658
kllrnohj wrote: post #627

Earlier quoted context omitted.

I mean "value types" https://www.jesperdj.com/2015/10/04/project-valhalla-value-t... & http://cr.openjdk.java.net/~jrose/values/values-0.html I don't know what you mean by "copy-only types." I'm not finding any reference to that terminology in the context of language design.

Ah, thanks for the link; I wasn't sure what it meant in the context of Java, since it's possible to get value semantics using a class. Sorry about the confusion. I meant for the quotes around "copy-only" to indicate that it wasn't really a standard term, but I marked "value types" the same way, so that didn't really work. By "copy-only" I meant something you couldn't have more than one reference to: every name (varia…

> By "copy-only" I meant something you couldn't have more than one reference to: every name (variable) to which you assign the data would have its own independent copy.

That's not really a requirement of value types, no. C# has value types (structs) and you can have references to them as well (ref & out params).

In general though yes it would be typically copied around, same as an 'int' is. Particularly if Java doesn't add something equivalent to ref/out params.

Re: Why Discord is switching from Go to Rust

#660

Post by Uristqwerty »

takeda wrote: post #648
Uristqwerty wrote: post #143

Earlier quoted context omitted.

I don't know about anyone else, but I like aligning certain things at half-indents (labels/cases half an indent back, so you can skim the silhouette of both the surrounding block and jump targets within it; braceless if/for bodies to emphasize their single-statement nature (that convention alone would have made "goto fail" blatantly obvious to human readers, though not helped the compiler); virtual blocks created by…

So you are the person that ruins it for everyone (are you emacs user by any chance?). Tabs are more versatile, you can even use proportional fonts with them. Projects end up using tabs because many people end up mixing them together (unknowingly or in your case knowingly using configuration that is unavailable in many IDEs). BTW when you mix spaces with tabs you eliminate all benefits that tabs give (for example you…

If I were an emacs user, I'd figure out how to write a plugin to display tab-indented code to my preferences.

No, I used to be a notepad user (on personal projects, not shared work) (you can kinda see it in the use of indentation to help convey information that IDEs would use font or colour to emphasize), and these days use tabs but longingly wish Eclipse, etc. had more options in their already-massive formatting configuration dialogues.

Post reply on HN