Why Discord is switching from Go to Rust • Hacker News Board

Re: Why Discord is switching from Go to Rust

#571

Post by terminaljunkid » Wed, Feb 05, 2020, 9:36 AM UTC

kerkeslager wrote: post #183

nemo1618 wrote: post #93

I wonder if it would be feasible to rewrite the LRU cache (either fully or in part) in a way that does not require the GC to scan the entire cache.

Yes, it's possible: that's generational garbage collection. But last I heard, Google decided writing a modern GC was too complicated. They're probably right, because Google doesn't need it. But for everyone else who decided to use a language designed to solve Google's fairly-unique problems as if it were a general-purpose language: that kind of sucks, doesn't it?

The fact seems to be that the go team is not so well funded as it seems. Go is not Google's language in the sense C# is MS' language or Java was Sun language.

Re: Why Discord is switching from Go to Rust

#572

Post by _ph_ » Wed, Feb 05, 2020, 9:38 AM UTC

Jweb_Guru wrote: post #557

_ph_ wrote: post #538

Earlier quoted context omitted.

I don't get, why this is downvoted without comments. Compared to a rewrite, this would have been a miniscule change. Furthermore, considering that you wrote that long blog post (which I quite appreciate, as it contains interesting information), it would have been important knowledge, whether the setting of the parameter was the real culprit - and if it was, a good reason to shout out to the Go implementors to look cl…

All I'm going to say is that if you think maintaining your own version of a compiler is the reasonable option compared to a rewrite in another language, you are probably deeply invested in the former language. This also applies to kernels and databases.

Well, in this case, "maintaining your own version compiler" concerns a single value change in the code base. At least, as I wrote, it should have been tried to identify the root cause for the observed behavior. If this "fix" significantly improves the behavior, it would have been a good data point to reach out to the Go developers to resolve this issue.

The problem to get down to the core of these issues are test cases. It seems, that neither the Go developers nor many other people have run into this as an issue - I only remember noticing the regular GC some years ago, but it was not an issue for me. As they have a real-life test case exposing this problem, they are possibly the only ones, who could verify a potential fix for the problem.

So, while it is great that they identified the problem and wrote a thorough blog piece about it, the only thing we learn from this is, that in Go 1.9 there was a latency issue every 2 minutes with their style of application/heap usage. Unfortunately, we don't know whether this problem was already addressed in later Go versions, and if not, whether there should be a way to control the automatic gc intervals to address this.

Re: Why Discord is switching from Go to Rust

#573

Post by _ph_ » Wed, Feb 05, 2020, 9:43 AM UTC

calcifer wrote: post #563

_ph_ wrote: post #536

Earlier quoted context omitted.

But on what basis? What part of Go isn't implemented in Go?

I gave an example a bit further up, here [1]. [1] https://news.ycombinator.com/item?id=22240223

But this isn't a contradiction to the statement, that Go is implemented in Go. If you look at the sources of the Go implementation, the source code is 99% Go, with a few assembly functions (most for optimizations not performed by the compiler) and no other programming language used.

Re: Why Discord is switching from Go to Rust

#574

Post by dom96 » Wed, Feb 05, 2020, 9:48 AM UTC

vips7L wrote: post #511

jholman wrote: post #505

Earlier quoted context omitted.

Maybe some big FAANG company. Start at the beginning of the acronym, I guess. I wonder if anyone could persuade anyone at Facebook to do a little bit of D professionally. I bet if even one serious Facebook engineer made a serious effort to use D, its adoption woes would be over.

Walter Bright the author of D worked at Facebook writing a fast preprocessor for C/C++ in D.

Walter too? AFAIK it was just Andrei.

Re: Why Discord is switching from Go to Rust

#575

Post by FisherGuy44 » Wed, Feb 05, 2020, 9:51 AM UTC

This is not a fair comparison. Go 1.9.2 was released over 2 years ago. In that time they have fixed a lot of the GC stutter issues. Comparing rust nightly to a 2 year old compiler is unfair.

Re: Why Discord is switching from Go to Rust

#576

Post by pkolaczk » Wed, Feb 05, 2020, 10:01 AM UTC

_ph_ wrote: post #264

robocat wrote: post #253

Earlier quoted context omitted.

You still may have significant CPU overhead from the GC e.g. the twitch article (mentioned elsewhere in comments) measured 30% CPU used for GC for one program (Go 1.5 I think). Obviously they consider spending 50% more on hardware is a worthwhile compromise for the gains they get (e.g. reduction of developer hours and reduced risk of security flaws or avoiding other effects of invalid pointers).

In this case, as they were running into the automatic GC interval, their program did not create much, if any garbage. So the CPU overhead for the GC would have been quite small. If you do a lot of allocations, the GC overhead rises of course, but also would the effort of doing allocations/deallocations with a manual managing scheme. In the end it is a bit trade-off, what fits the problem at hand best. The nice thing…

Languages that have GC frequently rely on heap allocation by default and make plenty of allocations. Languages with good manual memory management frequently rely on stack allocation and give plenty of tools to work with data on the stack. Automatic allocation on the stack is almost always faster than the best GC.

Re: Why Discord is switching from Go to Rust

#577

Post by rswail » Wed, Feb 05, 2020, 10:18 AM UTC

cma wrote: post #410

hu3 wrote: post #375

Earlier quoted context omitted.

Let's not fool ourselves. The demeaning of janitors was introduced by GP by describing it as something they would rather not do. No mental gymnastics required.

He said he felt like a janitor, next guy said he demeaned others as janitors, and now you are saying he demeaned janitors. There is a level of gymnastics going on.

I'm not a gymnast, but comparing people comparing their work to janitors and calling it gymnastics demeans gymnasts.

/s

Re: Why Discord is switching from Go to Rust

#578

Post by nindalf » Wed, Feb 05, 2020, 10:27 AM UTC

no_wizard wrote: post #508

systems wrote: post #462

Earlier quoted context omitted.

I strongly believe, that people should consider the ecosystem part of maintainability risk and claim Rust have by far the better ecosystem compared to those 3

That’s highly subjective claim to bare without evidence. All three languages are actively maintained and are growing. My refute it simply: Rusts web development story isn’t out of the box clean like Crystal Lang’s which ships with an HTML language out of the box. So it could be categorized as a poor choice in comparison to Crystal

> ships with an HTML language

Did you mean HTTP server? If so, there are at least 3 good ones in Rust that are only a `cargo add` away. If you've already taken the trouble to set up a toolchain for a new language, surely a single line isn't asking too much.

Re: Why Discord is switching from Go to Rust

#579

Post by barrkel » Wed, Feb 05, 2020, 10:34 AM UTC

arcticbull wrote: post #531

giornogiovanna wrote: post #524

Earlier quoted context omitted.

Reference counting is a (low-throughput, low-latency) form of garbage collection.

Yes and no. From a theoretical perspective, I suppose that's true, but "garbage collection" tends to mean a non-deterministic collector that does its own thing, and you don't have to think at all about memory. That does not apply to Swift, as of course, you need to understand the difference between strong and weak references. It's unfairly simplistic to couple the two.

No, RC is GC.

Most people think of Python as GCed language, and it uses mostly RC.

Any runtime that uses mark & sweep today may elect to use RC for some subset of the heap at some point in a future design, if that makes more sense. The mix of marking GC vs refcounting GC shouldn't affect the semantics of the program.

Re: Why Discord is switching from Go to Rust

#580

Post by barrkel » Wed, Feb 05, 2020, 10:37 AM UTC

arcticbull wrote: post #532

terminaljunkid wrote: post #455

Earlier quoted context omitted.

> Swift doesn't have a GC slowing things down This Apple marketing meme needs to die. Reference counting incurs arguably more cost than GC, or at least the cost is spread through-out processing.

Apple's ARC is not a GC in the classic sense. It doesn't stop the world and mark/sweep all of active memory. It's got "retain" and "release" calls automatically inserted and elided by the compiler to track reference counts at runtime, and when they hit zero, invoke a destructor. That's not even close to what most people think of when they think "gc". Of course it's not free, but it's deterministic.

You're using a hearsay, folklore definition of GC, not a CS one.

Refcounting in the presence of threads is usually non-deterministic too.