Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

451–460 of 670 posts

Re: Why Discord is switching from Go to Rust

#451

Post by buzzerbetrayed »

blazespin wrote: post #429

Confused, aren't they losing memory safety? I get for certain core code situations, you want to manage all memory safety yourself (or use built in static GC), but beyond that it seems to me at a higher level you'd rather have the automatic GC. Why burden all of your developers rather than just a core few? I don't think GC issues is a compelling argument to move everything to Rust. I'm not saying there aren't compelli…

I’ve never heard the argument that moving to rust reduces memory safety. Isn’t memory safety what rust is known for?

Re: Why Discord is switching from Go to Rust

#452

Post by millstone »

echelon wrote: post #426
marta_morena wrote: post #323

Earlier quoted context omitted.

Sorry, but `Much simpler and nicer` is something that I highly doubt when you talk about Java to Rust. Unless the people writing the Java code were C programmers, lol, in which case I feel for you.

I write Java during the day but wish I could write Rust instead. Rust has a much nicer standard library, and the semantics and abstractions are so incredibly beautiful. It's a language that has learned from the millions of human-years that went into other languages and ecosystems. Every corner and seam in the language design speaks to this. Traits, union type enums, pattern matching, option and result types, derive()…

Rust has its share of random-feeling nonsense, like requiring PhantomData to "work around" unused type params, or that you can't compare arrays longer than 32 elements.

Re: Why Discord is switching from Go to Rust

#453

Post by topspin »

ascv wrote: post #434
topspin wrote: post #291

Earlier quoted context omitted.

> It's surprising they didn't test upgrading to 1.13. It isn't surprising to me. It's stated elsewhere they tried 4 difference version of Go, up through 1.10 apparently, and had performance problems with all of them. At some point you can't suffer garbage collector nonsense anymore and since they'd already employed Rust on other services they tried it here. It worked on the first try. That's not surprising either. Wh…

It's still a huge whoosh. You're starting at 1.9 and you're testing 4 micro versions to 1.10.. what is the point of that? None of those non-major versions are going to significantly change how the GC works. They could have tried 1 other version (not 4) and picked either the latest (1.13) or the version that contains the GC improvements (1.12) to test. Usually when you are looking to upgrade something you skim the rel…

> 1.12 seems to specifically address their performance concern

Maybe it does, maybe it doesn't. Maybe 1.14 has a performance regression with their specific load and they get screwed.

We'll never know. You know why? They permanently solved their GC problems by eliminating GC.

That's the smart play.

Re: Why Discord is switching from Go to Rust

#454

Post by blackrock »

jerf wrote: post #257
blackrock wrote: post #204

Would it have been better if they went with Elixir? Write their code in a functional style. Get the benefits of the Erlang BEAM platform. Their system runs over the web, so time sensitivity isn’t as important, in comparison to video games, VR, or AR. Anyone ever done a performance comparison breakdown between something like Elixir vs. Rust?

"Would it have been better if they went with Elixir?" No. It would have been unshippably bad. BEAM is generally fairly slow. It was fast at multitasking for a while, but that advantage has been claimed by several other runtimes in 2020. As a language, it is much slower than Rust. Plus, if you tried to implement a gigantic shared cache map in Erlang/Elixir, you'd have two major problems: One is that you'd need huge ch…

Good points.

> It wants lots of little processes, not a small number of processes holding tons of data

Elixir/Erlang is good for handling a lot of little processes with a small amount of data. And not for a small number of processes, handling a large amount of data.

The little processes holds smaller data, and it just gets dropped, after the function is done, instead of getting reclaimed by a garbage collector.

This is probably what makes Elixir/Erlang good for telecom equipment, like packet switching hardware, but not good for more complex software applications that may need to fetch and manipulate a lot of structured data in multiple stages.

In this case, does anyone know of Elixir’s maximum throughput?

Re: Why Discord is switching from Go to Rust

#455

Post by terminaljunkid »

chubs wrote: post #307
dochtman wrote: post #256

Earlier quoted context omitted.

One the one hand, yes. On the other hand, all of this sounds much more complex and fragile. This seems like an important point to me: "Remarkably, we had only put very basic thought into optimization as the Rust version was written. Even with just basic optimization, Rust was able to outperform the hyper hand-tuned Go version."

I found similarly when I ported an image resizing algorithm from Swift to Rust: I'm experienced in swift thus was able to write in an idiomatic way, and have little Rust experience thus I wrote it in a naive way; yet still the rust algorithm was twice(!) as fast. And swift doesn't even have a GC slowing things down!

> Swift doesn't have a GC slowing things down

This Apple marketing meme needs to die. Reference counting incurs arguably more cost than GC, or at least the cost is spread through-out processing.

Re: Why Discord is switching from Go to Rust

#456

Post by latch »

_bxg1 wrote: post #86
wongarsu wrote: post #77

Earlier quoted context omitted.

Games written in the Unity engine are (predominately) written in C#, a garbage collected language. Keeping large amounts of data around isn't that unusual since reading from disk is often prohibitively slow, and it's normal to minimize memory allocation/garbage generation (using object pools, caches etc), and manually trigger the GC in loading screens and in other opportune places (as easy as calling System.GC.Collec…

Right; Go is purpose-built for writing web services, and web services tend to be pretty tolerant of (small) latency spikes because virtually anyone who's calling one is already expecting at least some latency

> Go is purpose-built for writing web services

Is this true? Go was built specifically for C++ developers, which, even when Go was first release, was a pretty unpopular language for writing web services (though maybe not at Google?). That a non-trivial number of Ruby/Python/Node developers switched was unexpected. (1)

https://commandcenter.blogspot.com/2012/06/less-is-exponenti...

Re: Why Discord is switching from Go to Rust

#457

Post by gridspy »

millstone wrote: post #452
echelon wrote: post #426

Earlier quoted context omitted.

I write Java during the day but wish I could write Rust instead. Rust has a much nicer standard library, and the semantics and abstractions are so incredibly beautiful. It's a language that has learned from the millions of human-years that went into other languages and ecosystems. Every corner and seam in the language design speaks to this. Traits, union type enums, pattern matching, option and result types, derive()…

Rust has its share of random-feeling nonsense, like requiring PhantomData to "work around" unused type params, or that you can't compare arrays longer than 32 elements.

1. I've never had to use PhantomData since I started coding in rust pretty much fulltime 2 years ago. 2. You can compare arrays longer than 32, but the compiler will no longer create an automatic comparison for you. So it's not that you "can't" do it. -- That said, using == to compare >32 elements sounds inefficient, perhaps check your use-case?

Re: Why Discord is switching from Go to Rust

#458

Post by [deleted] »

echelon wrote: post #426
marta_morena wrote: post #323

Earlier quoted context omitted.

Sorry, but `Much simpler and nicer` is something that I highly doubt when you talk about Java to Rust. Unless the people writing the Java code were C programmers, lol, in which case I feel for you.

I write Java during the day but wish I could write Rust instead. Rust has a much nicer standard library, and the semantics and abstractions are so incredibly beautiful. It's a language that has learned from the millions of human-years that went into other languages and ecosystems. Every corner and seam in the language design speaks to this. Traits, union type enums, pattern matching, option and result types, derive()…

[deleted]

Re: Why Discord is switching from Go to Rust

#459

Post by Symmetry »

nicoburns wrote: post #289
Symmetry wrote: post #202

Earlier quoted context omitted.

I think the way I'd put it is that languages with manual memory management, like Rust, have more scope for optimization than languages that don't. You can just use the gc crate in Rust and have almost the same ease of development but the same performance problems you do in Go.

Notably, it didn't sound like the development in Rust was particularly difficult anyway. That's certainly been my experience in any case.

It also sounded like the developers had already thought carefully about their memory usage patterns and had been optimizing for performance as much as they could within the scope Go allowed them. Personally I've found Rust has a higher cognitive overhead than Go when I'm just banging something out only worrying about correctness but if you're thinking carefully about memory usage patterns in a way you need to to optimize performance there's no penalty.
Post reply on HN