Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Taming Go’s memory usage, or how we avoided rewriting our client in Rust

akitasoftware.com

Post reply on HN
Print view View original

61–70 of 231 posts

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#61

Post by vp8989 »

sreque wrote: post #53
fiddlerwoaroof wrote: post #42

Earlier quoted context omitted.

I’m not really sure if that 2x figure is accurate. I’ve seen charts on both sides of this and a lot here depends on your programming language and the things it can optimize: with Linear/Affine types, I’m fairly sure Haskell could, in theory, eliminate GC deterministically from the critical sections of your code-base without forcing you to adopt manual memory management universally. But, there’s just the fact that peo…

I am aware that you can hit really good latency targets with GC'ed languages, like in the video game and finance industry. Whenever I investigate examples, though, I find the devs have to go through a ton of effort to avoid memory allocations, and then I ask if using the GC'ed language was even worth it in the first place? I'm actually fascinated with the idea of going off-heap in the hotspots of GC'ed languages to g…

"Whenever I investigate examples, though, I find the devs have to go through a ton of effort to avoid memory allocations"

Yep, also the median dev in a GC'ed language is simply incapable of writing super efficient code in these languages because they rarely have to. You would have to bring in the best of the best people from those communities or put your existing devs through a pretty significant education process that is similar in difficulty to just learning/using Rust.

The resulting code will be very different to what typical code looks like in those languages, so the supposed homogeneity benefits of just writing fast C#/Java when it's needed are probably not quite true. You'd basically have to keep that project staffed up with these kinds of people and ensure they have very good Prod observability to ensure regressions don't appear.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#63

Post by sreque »

vp8989 wrote: post #61
sreque wrote: post #53

Earlier quoted context omitted.

I am aware that you can hit really good latency targets with GC'ed languages, like in the video game and finance industry. Whenever I investigate examples, though, I find the devs have to go through a ton of effort to avoid memory allocations, and then I ask if using the GC'ed language was even worth it in the first place? I'm actually fascinated with the idea of going off-heap in the hotspots of GC'ed languages to g…

"Whenever I investigate examples, though, I find the devs have to go through a ton of effort to avoid memory allocations" Yep, also the median dev in a GC'ed language is simply incapable of writing super efficient code in these languages because they rarely have to. You would have to bring in the best of the best people from those communities or put your existing devs through a pretty significant education process th…

Yes, and I think one important aspect to this is the necessary CI/CD changes needed to support these kinds of optimizations. If your performance targets are tight enough that you are making significant non-standard optimizations in your GC'ed language, you're probably going to want some automated performance regression testing in your deployment pipeline to ensure you don't ship something that falls down under load. In my experience, building and maintaining those pipeline components is not easy.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#64

Post by saghm »

notamy wrote: post #2

Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time…

There are already a lot of replies to this comment explaining the ideas behind Rust memory management in different ways, but I'll throw in my handwavy explanation as well:

In GC languages, memory management is generally runtime through the interpreter/runtime. In C, memory management is generally done at programming time by the (human) programmer. In Rust, memory management is generally done at compile time by the compiler. There are exceptions in all three cases, but the "default" paradigm of a language informs a lot about how it's designed and used.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#65

Post by laumars »

xondono wrote: post #15
wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

> Buried in here are great examples of why rewrites don’t help That has not been my experience. Rewrites do sometimes help, because in a lot of codebases there’s too many “pet” modules or badly designed frozen interfaces. Rewrites can help in those situations, because there’s no sacred cows anymore. The issue is that a lot of people do rewrites as translations, without touching structures.

This is where profiling helps more. Find the weak parts of the code, try to optimise those. If the language proves to be a barrier then you have a justification for a rewrite.

All too often people don’t understand how to performance tune software properly and instead blame other things first (eg garbage collection)

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#66

Post by pvg »

titzer wrote: post #60
tptacek wrote: post #39

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were usin…

> Frankly I'm surprised Go acquitted itself as well as it did here. As opposed to, e.g. Java, which I ranted elsewhere in the thread, is a trashy mess. I programmed for over a decade in Java, and yeah, it's only gotten worse over the years. They would have done even more custom processing and bypassing of the layers underneath due to Java's typical copy-happiness.

This kind of analysis and remediation would work just as well in Java and is often a more rigorous and effective approach than the author's somewhat Java-inspired initial idea of fiddling with GC parameters.

One big difference is that the Java runtime design intent is more in the vein of 'converting memory into performance'. On HN, Ron Pressler ('pron) has written a bunch of interesting stuff about that over the years

https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#67

Post by bluGill »

xondono wrote: post #15
wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

> Buried in here are great examples of why rewrites don’t help That has not been my experience. Rewrites do sometimes help, because in a lot of codebases there’s too many “pet” modules or badly designed frozen interfaces. Rewrites can help in those situations, because there’s no sacred cows anymore. The issue is that a lot of people do rewrites as translations, without touching structures.

Last time I was in a rewrite the boss had the old software on a computer next to him with the label "Product owner of rewrite". He regularly when asked how to do something looked at what that did.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#68

Post by throwaway894345 »

steveklabnik wrote: post #5
notamy wrote: post #2

Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time…

Yes, Rust kinda doesn't fit super cleanly into a very black/white binary here. It is automatic in the sense that you do not generally call malloc/free. The compiler handles this for you. At the same time, you have a lot more control than you do in a language with a GC, and so to some people, it feels more manual. It's also like, a perception thing in some sense. Imagine someone writes some code. They get a compiler e…

"Manual vs automatic" is mostly just a semantic problem IMHO. We could say "runtime versus compile time" to be more precise, but maybe there are problems there as well. The more interesting question to me is "how much time/energy do I spend thinking about memory management, and is that how my time is best spent?". In cases of high performance code, you might spend more time fighting with the GC than you would with the borrow checker to get the performance you need, but for everything else the hot paths are so few and far between you're most likely better off fighting with the GC 1% of the time and not fighting anything the other 99%.

The Rust community has done laudable work in bringing down the cognitive threshold of "manual / compile-time" memory management, but I think we're finding out that the returns are diminishing quickly and there's still quite a chasm between borrow checking and GC with respect to developer velocity.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#69

Post by bluGill »

laumars wrote: post #65
xondono wrote: post #15

Earlier quoted context omitted.

> Buried in here are great examples of why rewrites don’t help That has not been my experience. Rewrites do sometimes help, because in a lot of codebases there’s too many “pet” modules or badly designed frozen interfaces. Rewrites can help in those situations, because there’s no sacred cows anymore. The issue is that a lot of people do rewrites as translations, without touching structures.

This is where profiling helps more. Find the weak parts of the code, try to optimise those. If the language proves to be a barrier then you have a justification for a rewrite. All too often people don’t understand how to performance tune software properly and instead blame other things first (eg garbage collection)

Most slow languages make escape to C easy for cases where the language is the issue. Most fast languages make writing a C APIed interface easy, so if the language is your issue just rewrite the parts where that is the problem.

Of course eventually you get to the point where enough of the code is in a fast language that writing everything in the fast language to avoid the pain of language interfaces is worth it.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#70

Post by tptacek »

kevingadd wrote: post #44
tptacek wrote: post #39

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were usin…

Reflection APIs seem to be pretty messy and slow in every runtime I've ever used, perhaps because the idea of optimizing them might encourage more use. The C# reflection APIs also allocate a lot.

A thing you can ding Go for is that you can find yourself relying on `reflect` (under the hood) more than you expect, because it's how you do things like read struct tags for things like JSON.

But that's not what the problem was here; the product they were building was using `reflect` in anger. They were relying on something that did magic, pulling a rabbit out of its hat to automatically compare protobuf thingies. They used it on a hot path. The room quickly filled with rabbit corpses. I guess you can blame Go for the existence of those kinds of libraries, but most perf-sensitive devs know that they're a risk.

Post reply on HN