Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Discord is switching from Go to Rust

blog.discordapp.com

Post reply on HN
Print view View original

551–560 of 670 posts

Re: Why Discord is switching from Go to Rust

#551

Post by PeterCorless »

the-alchemist wrote: post #96

Looks like the big challenge is managing a large, LRU cache, which tends to be a difficult problem for GC runtimes. I bet the JVM, with its myriad tunable GC algorithms, would perform better, especially Shenandoah and, of course, the Azul C4. The JVM world tends to solve this problem by using off-heap caches. See Apache Ignite [0] or Ehcache [1]. I can't speak for how their Rust cache manages memory, but the thing to…

Wow. We literally just published why to not put a cache in front of your server to mask its bad performance behind a layer of complexity. tl;dr: make sure you have a solid DB to begin with. (Forgive the gated asset, but it's a good read!)

https://go.scylladb.com/7-reasons-no-external-cache-database...

Re: Why Discord is switching from Go to Rust

#552

Post by Jweb_Guru »

Matthias247 wrote: post #510
buzzerbetrayed wrote: post #451

Earlier quoted context omitted.

I’ve never heard the argument that moving to rust reduces memory safety. Isn’t memory safety what rust is known for?

It is! But in Rust you still have an escape hatch in the form of the `unsafe` annotation which allows for mistakes which break memory safety. I don't think Go has something like that, unless you use the FFI. So saying that Go is at least as memory safe as Rust might not be too wrong of a statement. However I think in total Rust is safer. E.g. Rust prevents a ton of race conditions in multithreaded code, which Go can…

Go has data races on multiple cores in safe code, without using any unsafe intrinsics or C FFI.

Re: Why Discord is switching from Go to Rust

#553

Post by Jweb_Guru »

scott_s wrote: post #273
dennisgorelik wrote: post #228

> Changing to a BTreeMap instead of a HashMap in the LRU cache to optimize memory usage. Why would BTreeMap be faster than HashMap? HashMap performance is O(1), while BTreeMap performance is O(log N).

This subthread explains why it's more memory efficient to use a tree-based structure: https://news.ycombinator.com/item?id=22239393 . Short version is that in order to get good performance out of a hashtable based structure, you want to have more than n slots in order to achieve good performance. Which brings me to my second point: hashtable based data structures are not worst-case O(1) . They are worst-case O(n) , b…

Cuckoo hashmaps are worst case O(1) when implemented correctly, up to resizing (however, they do need more space and perform worse in virtually all real benchmarks).

Re: Why Discord is switching from Go to Rust

#554

Post by fxtentacle »

Sounds like badly reinventing the wheel. If you need a large in-memory LRU cache, use memcached. Problem solved, because then Go doesn't need to allocate much memory anymore. And I'd wager that JSON serialization for sending a reply back to the client will dominate CPU load anyway, so that the overhead for Go to talk to Memcached will be barely noticeable.

Re: Why Discord is switching from Go to Rust

#555

Post by Jweb_Guru »

millstone wrote: post #452
echelon wrote: post #426

Earlier quoted context omitted.

I write Java during the day but wish I could write Rust instead. Rust has a much nicer standard library, and the semantics and abstractions are so incredibly beautiful. It's a language that has learned from the millions of human-years that went into other languages and ecosystems. Every corner and seam in the language design speaks to this. Traits, union type enums, pattern matching, option and result types, derive()…

Rust has its share of random-feeling nonsense, like requiring PhantomData to "work around" unused type params, or that you can't compare arrays longer than 32 elements.

It's not really a workaround (it didn't used to be required), it asks for PhantomData so it can figure out variance and infer auto traits.

Re: Why Discord is switching from Go to Rust

#556

Post by staticassertion »

arcticbull wrote: post #531
giornogiovanna wrote: post #524

Earlier quoted context omitted.

Reference counting is a (low-throughput, low-latency) form of garbage collection.

Yes and no. From a theoretical perspective, I suppose that's true, but "garbage collection" tends to mean a non-deterministic collector that does its own thing, and you don't have to think at all about memory. That does not apply to Swift, as of course, you need to understand the difference between strong and weak references. It's unfairly simplistic to couple the two.

But it actually is just garbage collection.

Re: Why Discord is switching from Go to Rust

#557

Post by Jweb_Guru »

_ph_ wrote: post #538
_ph_ wrote: post #259

Earlier quoted context omitted.

Yes, you have to rebuild go, but that is literally done in a minute. It also would be interesting, if you happen to have some conclusive benchmarks, how the latest Go runtime would perform in this sense.

I don't get, why this is downvoted without comments. Compared to a rewrite, this would have been a miniscule change. Furthermore, considering that you wrote that long blog post (which I quite appreciate, as it contains interesting information), it would have been important knowledge, whether the setting of the parameter was the real culprit - and if it was, a good reason to shout out to the Go implementors to look cl…

All I'm going to say is that if you think maintaining your own version of a compiler is the reasonable option compared to a rewrite in another language, you are probably deeply invested in the former language. This also applies to kernels and databases.

Re: Why Discord is switching from Go to Rust

#558

Post by cies »

cdoxsey wrote: post #515
cies wrote: post #347

Earlier quoted context omitted.

As the grant parent commentor i cannot down vote, so that wasn't me. Google explicitly shown no intent to make Go a fit beyond network apps. You can hack something into doing more than originally intended, but then you are usually operating "outside of warranty". > On the other hand, you could argue that Rust is a wrong fit for anything _except_ performance-critical applications Well, Rust does more than C-level high…

Go is a general purpose programming language. It is suitable for a large variety of programming tasks beyond network services (though itself a massive problem domain) For example Go is great for building cli apps. Simple, easy to install and easy to understand. For another Go has surprisingly good windows support. Google didn’t do that. For a third Go has robust cryptography libraries. There are actually lot’s of oth…

> Go is a general purpose programming language.

Seriously? Does Google say that "native GUI devt" is intended use? And how about that it only supports one concurrency method, and does not allow one to implement one yourself.

> For example Go is great for building cli apps.

Ok, your joking right? CLI apps are simply a networked app that does not necessarily use the network. That's not an entirely new domain, like OpenGL, native GUIs, embedded systems, kernel programming, ...

> There are actually lot’s of other contributions from the community if you’d take the time to look.

That's good, but it still is not "open innovation" to the level of say Rust.

Re: Why Discord is switching from Go to Rust

#559

Post by abjKT26nO8 »

pkolaczk wrote: post #530
terminaljunkid wrote: post #455

Earlier quoted context omitted.

> Swift doesn't have a GC slowing things down This Apple marketing meme needs to die. Reference counting incurs arguably more cost than GC, or at least the cost is spread through-out processing.

It incurs some cost, but whether it is higher is very debatable. This is very much workload dependent. A smart compiler can elide most reference updates.

It would seem that the Swift compiler is far from smart[1].

[1]: " rel="nofollow">https://media.ccc.de/v/35c3-9670-safe_and_secure_drivers_in_...

Re: Why Discord is switching from Go to Rust

#560

Post by tcbasche »

Tomis02 wrote: post #490
SlowRobotAhead wrote: post #469

Earlier quoted context omitted.

This is what clicked for me on microservices years back. That the language wasn’t important and if I couldn’t do it in python or C, someone else could in Go or Java or etc. Compared to if I wrote something in house entirely in C... lolno

Landing in a shop that uses N programming languages for N microservices would be a pretty miserable experience.

At a previous job we used Python for all microservices, except for 'legacy' systems which were in Groovy / Rails. That was a context switch if I ever experienced one.
Post reply on HN