Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Taming Go’s memory usage, or how we avoided rewriting our client in Rust

akitasoftware.com

Post reply on HN
Print view View original

21–30 of 231 posts

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#21

Post by coliveira »

wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

That is correct.

In the worst case, you can always (even on GC'd languages) pre-allocate buffers and do your work without new memory requests. But you need to plan for this, in the same way you'd do in a language without GC.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#22

Post by coliveira »

xondono wrote: post #15
wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

> Buried in here are great examples of why rewrites don’t help That has not been my experience. Rewrites do sometimes help, because in a lot of codebases there’s too many “pet” modules or badly designed frozen interfaces. Rewrites can help in those situations, because there’s no sacred cows anymore. The issue is that a lot of people do rewrites as translations, without touching structures.

This is less an argument for a rewrite than an argument for redesigning parts of your codebase, which can be done much more easily than a complete rewrite.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#23

Post by olau »

wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

> The problem here isn’t that the language has GC, it’s that memory usage was just not considered.

While I agree with the gist of what you're saying, I do think runtimes based on the we'll-clean-it-up-some-day GC paradigm makes it more important to consider memory allocation than less laissez-faire paradigms (like RAII or reference counting), contrary to how it's presented in the glamorous brochures.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#24

Post by slaymaker1907 »

notamy wrote: post #2

Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time…

While some commenters have pointed out that you still need to deal with lifetimes/thinking about where stuff lives, in practice you can avoid almost all of this by using Rc instead of Type everywhere (or Arc in a multithreaded scenario).

Yes Rc and equivalents have a performance overhead, but for many use cases the overhead really isn't that bad since you typically aren't creating tons of copies. In practice, I've found one can ignore lifetimes in almost all cases even when using references except when storing them in structs or closures. So really you would just need to increment the Rc counter for structs/closures outside of allocation/deallocation which is dominated by calls to malloc/free.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#25

Post by sreque »

wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

I downvoted you at first and then changed my mind. I think I would like your comment more if it were more worded like: "buried in here are great examples of important optimizations that did not require a rewrite". Or something like: "this article does a great job of showing that you can hit many reasonable performance targets while using a GC'ed language like Go."

You can pretty much always get better performance with more control over memory, and more importantly, you can dramatically lower overall memory usage and avoid GC pauses, but you have to weigh that against the fact that automated memory management is one of the few programming language features that is basically proven to give a massive developer productivity boost. In my corner of the industry, everyone chooses the GC'ed languages and performance isn't really a major concern most of the time.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#26

Post by jcelerier »

Spartan-S63 wrote: post #3
notamy wrote: post #2

Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time…

I feel that this is one of those common misconceptions about Rust. Rust's memory management is nothing like C or non-modern C++'s with malloc/free or new/delete. Rust uses modern-C++'s RAII model, typically, to allocate memory. The compiler is smart enough to know when to call drop() (which is essentially free/delete, but with the possibility of additional behavior). You can also call drop() yourself. What I think pe…

It kills me that RAII is considered modern c++. It's there since 1983 aha, what do you think fstream and std::vector are if not RAII wrappers over files or memory

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#27

Post by brink »

Spartan-S63 wrote: post #3
notamy wrote: post #2

Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time…

I feel that this is one of those common misconceptions about Rust. Rust's memory management is nothing like C or non-modern C++'s with malloc/free or new/delete. Rust uses modern-C++'s RAII model, typically, to allocate memory. The compiler is smart enough to know when to call drop() (which is essentially free/delete, but with the possibility of additional behavior). You can also call drop() yourself. What I think pe…

> Additionally, Rust is pretty in-your-face when it comes to concurrency and sharing memory across thread/task boundaries.

Use channels whenever possible.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#28

Post by jjoonathan »

wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

Right, but GC encourages you to not think about memory at all until the program starts tipping over and fixing the underlying cause of the leak now requires an architecture change because the "we hold onto everything" assumption got baked into the structure in 2 places that you know about and 5 that you don't.

I don't miss the rote parts of manual memory management, but it had the enormously beneficial side effect of making people consider object lifetimes upfront (to keep the retain graph acyclic) and cultivate occasional familiarity with leak tracking tools. Problematic patterns like the undo queue or query correlator that accidentally leak everything tended to become obvious when writing the code, rather than while running it. These days, I keep seeing those same memory management anti-patterns show up when I ask interviewees to tell a debugging war story. Sometimes I even see otherwise capable devs shooting in the dark and missing when it comes to the "what's eating RAM" problem.

I feel like GC in long-form program development substitutes a small problem for a big one. Short-form programming can get away with just leaking everything, which is what GC does anyway, so I'm not sure there's any benefit there either.

tl;dr: get off my lawn.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#29

Post by CraigJPerry »

> For our application, it would be acceptable to simply exit when memory usage gets too large

Could you not just set a ulimit on memory usage of the process in that case? (And use another process as the parent, e.g. a supervisor or init, to avoid exiting the container and just restart the process instead)

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#30

Post by pjc50 »

notamy wrote: post #2

Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time…

You can also kind of do your own management of memory in GC languages, you just have to be extremely careful in code review to spot inadvertant allocations in the hot path. A great example is the "LMAX Disruptor" in Java: https://lmax-exchange.github.io/disruptor/

The trick is to pre-allocate all your objects and buffers and reuse them in a ring buffer. Similar techniques work in zero-malloc embedded C environments.

Post reply on HN