Taming Go’s memory usage, or how we avoided rewriting our client in Rust • Hacker News Board

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#51

Post by [deleted] » Tue, Sep 21, 2021, 8:07 PM UTC

sreque wrote: post #25

wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

I downvoted you at first and then changed my mind. I think I would like your comment more if it were more worded like: "buried in here are great examples of important optimizations that did not require a rewrite". Or something like: "this article does a great job of showing that you can hit many reasonable performance targets while using a GC'ed language like Go." You can pretty much always get better performance wit…

[deleted]

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#52

Post by jjoonathan » Tue, Sep 21, 2021, 8:07 PM UTC

tptacek wrote: post #34

jjoonathan wrote: post #28

Earlier quoted context omitted.

Right, but GC encourages you to not think about memory at all until the program starts tipping over and fixing the underlying cause of the leak now requires an architecture change because the "we hold onto everything" assumption got baked into the structure in 2 places that you know about and 5 that you don't. I don't miss the rote parts of manual memory management, but it had the enormously beneficial side effect of…

Plenty of C programs do the equivalent of ioutil.ReadAll; it's not a GC thing.

"Leak everything because we can get away with it here" is a fine memory management strategy. "Why does my program keep getting killed?" isn't.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#53

Post by sreque » Tue, Sep 21, 2021, 8:09 PM UTC

fiddlerwoaroof wrote: post #42

sreque wrote: post #32

Earlier quoted context omitted.

More importantly, GC'ed languages tend to use at least 2x the memory of un-GC'ed languages and have to deal with the consequences of GC-induced pauses and generally inferior native code interop. Whether that matters to you or not depends on your application. No one is going to use a GC'ed language in the Linux Kernel, but practically 100% of backend applications are written in GC'ed languages because the productivity…

I’m not really sure if that 2x figure is accurate. I’ve seen charts on both sides of this and a lot here depends on your programming language and the things it can optimize: with Linear/Affine types, I’m fairly sure Haskell could, in theory, eliminate GC deterministically from the critical sections of your code-base without forcing you to adopt manual memory management universally. But, there’s just the fact that peo…

I am aware that you can hit really good latency targets with GC'ed languages, like in the video game and finance industry. Whenever I investigate examples, though, I find the devs have to go through a ton of effort to avoid memory allocations, and then I ask if using the GC'ed language was even worth it in the first place?

I'm actually fascinated with the idea of going off-heap in the hotspots of GC'ed languages to get better performance. Netty, for instance, relies on off-heap allocations to achieve better networking performance. But, once you do so, you start incurring the disadvantages of languages like C/C++, and it can get complicated mixing the two styles of code.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#54

Post by jjoonathan » Tue, Sep 21, 2021, 8:12 PM UTC

titzer wrote: post #35

jjoonathan wrote: post #28

Earlier quoted context omitted.

Right, but GC encourages you to not think about memory at all until the program starts tipping over and fixing the underlying cause of the leak now requires an architecture change because the "we hold onto everything" assumption got baked into the structure in 2 places that you know about and 5 that you don't. I don't miss the rote parts of manual memory management, but it had the enormously beneficial side effect of…

GC will not fix trashy programming. The problem is that many GC'd languages have adopted a style guide that commits to a lot of unnecessary allocations. For example, in Java, you can't parse an integer out of the middle of a string without allocating in-between. Ditto with lots of other common operations. Java has oodles of trashy choices. With auto-boxing, allocations are hidden. Without reified (let's say, type-spe…

> many GC'd languages have adopted a style guide that commits to a lot of unnecessary allocations.

Oh, that too. I forgot to rant about that.

> Virgil

Unfortunately I'd rather live with a crummy language that has strong ecosystem, tooling, and developer availability, so I'll never really know. It does sound nice, though.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#55

Post by hinkley » Tue, Sep 21, 2021, 8:13 PM UTC

aidenn0 wrote: post #49

kevingadd wrote: post #44

Earlier quoted context omitted.

Reflection APIs seem to be pretty messy and slow in every runtime I've ever used, perhaps because the idea of optimizing them might encourage more use. The C# reflection APIs also allocate a lot.

Before writing Clojure, Rich Hickey wrote FOIL[1], which used sockets to communicate between common lisp and the JVM (or CLR). When asked about making it in-process, Rich observed that the reflection overhead on the JVM was often as large, or larger, than the serialization overhead, so the gains to be had were limited. 1: http://foil.sourceforge.net/

From what I recall, the Java team copped to the intentionally slow accusation, but that started to change when they decided to embrace the notion of other languages besides Java running on the JVM. Unfortunately that would have been shortly after Clojure was born. It took a few releases for them to really improve that situation, and that was still shortly before they started doing faster releases.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#56

Post by pjmlp » Tue, Sep 21, 2021, 8:16 PM UTC

titzer wrote: post #35

jjoonathan wrote: post #28

Earlier quoted context omitted.

Right, but GC encourages you to not think about memory at all until the program starts tipping over and fixing the underlying cause of the leak now requires an architecture change because the "we hold onto everything" assumption got baked into the structure in 2 places that you know about and 5 that you don't. I don't miss the rote parts of manual memory management, but it had the enormously beneficial side effect of…

GC will not fix trashy programming. The problem is that many GC'd languages have adopted a style guide that commits to a lot of unnecessary allocations. For example, in Java, you can't parse an integer out of the middle of a string without allocating in-between. Ditto with lots of other common operations. Java has oodles of trashy choices. With auto-boxing, allocations are hidden. Without reified (let's say, type-spe…

Yeah, but that was one of Java's 1.0 mistakes, that thankfully Go, .NET, D, Swift, among others, did not make.

Now lets see if Valhalla actually happens.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#57

Post by pjmlp » Tue, Sep 21, 2021, 8:19 PM UTC

oconnor663 wrote: post #46

notamy wrote: post #2

Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time…

I think sometimes the "compiler manages memory for you" concept gets overplayed a bit. It's not as complex as that description makes it sound. If you understand C++ destructors, it's really the same thing. Objects get destroyed when they go out of scope, and any memory or other resources they own get freed. The differences come up when you look at what happens when you make a mistake, like holding a pointer to a free…

Try to implement a data structure that works across async runtimes, or a couple of GUI widgets, then you will get the point why some of us complain about the borrow checker, even with decades of experience in C and C++.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#58

Post by hinkley » Tue, Sep 21, 2021, 8:19 PM UTC

xondono wrote: post #15

wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

> Buried in here are great examples of why rewrites don’t help That has not been my experience. Rewrites do sometimes help, because in a lot of codebases there’s too many “pet” modules or badly designed frozen interfaces. Rewrites can help in those situations, because there’s no sacred cows anymore. The issue is that a lot of people do rewrites as translations, without touching structures.

I would argue that the rewrites help when the information architecture for the original code is proven to be wrong, and there is either no way to refactor the old code to the new model, or employee turnover has resulted in nobody having an emotional attachment to the old code.

That said, to slot in a new implementation you often have to make the external API very similar to the old one, which can complicate making the improvements you're after.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#59

Post by option_greek » Tue, Sep 21, 2021, 8:22 PM UTC

I have a feeling that they will end up eventually rewriting this in Rust as the use case they describe is where a non GC language can definitely provide more performance (beyond the case they solved). APM tools usually need to be more performant to ensure they add as little overhead to the actual service as possible. I guess what's helping here is that this is passive monitoring which allows a little lag in the system. Question relavent here is will there be more issues with memory in general based on their current roadmap.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#60

Post by titzer » Tue, Sep 21, 2021, 8:22 PM UTC

tptacek wrote: post #39

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were usin…

> Frankly I'm surprised Go acquitted itself as well as it did here.

As opposed to, e.g. Java, which I ranted elsewhere in the thread, is a trashy mess. I programmed for over a decade in Java, and yeah, it's only gotten worse over the years. They would have done even more custom processing and bypassing of the layers underneath due to Java's typical copy-happiness.