Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Taming Go’s memory usage, or how we avoided rewriting our client in Rust

akitasoftware.com

Post reply on HN
Print view View original

101–110 of 231 posts

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#101

Post by Zababa »

pjmlp wrote: post #87
brundolf wrote: post #82

Earlier quoted context omitted.

It's not the first lever an engineer should reach for regardless of the languages involved. Calling out Rust specifically feels like a bit of a cheap shot

To be fair, that is now the common "I rewrote X in Y" theme, which followed upon the Y ∈ { Ruby, Clojure, Scala, Kotlin,.... } from previous years.

And Go too! It's always fun to see posts from around 2014/2015 complaining about how every submission to Hacker News is now "I wrote X in Go", while now Go is the boring stuff and Rust is the hot new thing. I wonder what will be the next Rust though.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#102

Post by hinkley »

bluGill wrote: post #71
hinkley wrote: post #58

Earlier quoted context omitted.

I would argue that the rewrites help when the information architecture for the original code is proven to be wrong, and there is either no way to refactor the old code to the new model, or employee turnover has resulted in nobody having an emotional attachment to the old code. That said, to slot in a new implementation you often have to make the external API very similar to the old one, which can complicate making th…

> there is either no way to refactor the old code to the new model That doesn't happen. Write facades as needed. Even if they are slower than everything else write the facades so you can keep in production all along.

If you get the object ownership and the internal state model wrong (information architecture) facades don't help you.

You can't put an idempotent or pure functional wrapper around a design that isn't re-entrant and expect anything good to come from it. IF you get it to work, it'll be dog slow.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#103

Post by jrockway »

tptacek wrote: post #39

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were usin…

Agree strongly here. These are common sources of memory leaks in any language, and it's very likely that rewriting this code in Rust would lead to the exact same problems. (Other cases on HN, like Discord's in-memory cache and Twitch's "memory ballast" thing, are pretty Go specific -- the identical C program wouldn't have those particular bugs. But, the Go developers read these incident reports and do fix the underlying causes; I think Twitch's need for the "memory ballast" got fixed a few years ago, but well after the "don't use Go for that" meme was popularized.)

Buffering is a pretty common bad habit. As programmers, we know stuff is going to go wrong, and we don't want to tell the user "come back later" (or in this case, undercount TCP stream metrics)... we want to save the data and automatically process it when we can so they don't have to. But, unfortunately it's an intrinsic Law Of The Universe that if data comes in a X bytes per second, and leaves at X-k bytes per second, then eventually you will use all storage space in the Universe for your buffer, and then you have the same problem you started with. (Storage limits in mirror may be closer than they appear.) Getting it into your mind that you have to apply back pressure when the system is out of its design specification is pretty crucial. Monitor it, alert on it, fix it, but don't assume that X more bytes of RAM will solve your problem -- there will eventually be a bigger event that exceeds those bounds.

Incidentally, the reason why you can make Zoom calls and use SSH while you download a file is because people added software to your networking stack that drops packets even though buffer space in your consumer-grade router are available. That tells your download to chill out so SSH and video conferencing packets get a chance to be sent to the network. The people that made the router had one focus -- get the highest possible Speedtest score. Throughput, unfortunately, comes at the cost of latency (bandwidth * buffer size for every single packet!), and it's not the right decision overall.

I don't know where I was going with this rant but ... when your system is overloaded, apply backpressure to the consumers. A packet monitoring system can't do that (people wouldn't accept "monitoring is overloaded, stop the main process"), but it does have to give up at some point. If you don't have any more memory to reassemble TCP connections, mark the stream as an error and give up. If you're dumping HTTP requests into a database, and the database stops responding, you'll just have to tell the HTTP client at the other end "too many requests" or "temporarily unavailable". To make the system more reliable, keep an eye on those error metrics and do work to get them down. Don't just add some buffers and cross your fingers; you'll just increase latency and still be paged to fight some fire when an upstream system gets slow ;)

Edit to add: I have a few stories here. One of them is about memory limits, which I always put on any production service I run. sum(memory limits) < sum(memory installed in the machine), of course. One time I had Prometheus running in a k8s cluster, with no memory limit. Sometimes people would run queries that took a lot of RAM, and there was often slack space on the machine, so nothing bad happened. Then someone's mouse driver went crazy, and they opened the same Grafana tab thousands of times. On a high memory query. Obviously, Prometheus used as much RAM as it could, and Linux started OOM killing everything. Prometheus died, was rescheduled on a healthy node, and the next group of tabs killed it. Eventually, the OOM killer had killed the Kubelet on every node, and no further progress could be made. The moral of the story is that it would have been better to serve that user 1000 "sorry, Prometheus died horribly and we can't serve your request right now", which memory limits would have achieved. Instead, we used up all the RAM in the Universe to try to satisfy them, and still failed. (What was the resolution? I think we killed the bad browser, which happened to be a dashboard-displaying TV next to our desks. Then kubelets restarted, and I of course updated Prometheus to have a 4G memory limit. Retried 1000 tabs with an expensive query, and Prometheus died and the frontend proxy served 990 of the tabs an error message. Back pressure! It works! You can imagine how fun this story would have been if I had cluster autoscaling, though. Would have just eventually come back to a $1,000,000 AWS bill and a 1000 node Kubernetes cluster ;)

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#104

Post by tptacek »

Zababa wrote: post #101
pjmlp wrote: post #87

Earlier quoted context omitted.

To be fair, that is now the common "I rewrote X in Y" theme, which followed upon the Y ∈ { Ruby, Clojure, Scala, Kotlin,.... } from previous years.

And Go too! It's always fun to see posts from around 2014/2015 complaining about how every submission to Hacker News is now "I wrote X in Go", while now Go is the boring stuff and Rust is the hot new thing. I wonder what will be the next Rust though.

BPF-verified C.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#105

Post by pjmlp »

arsome wrote: post #17
henning wrote: post #14

> Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time to manage memory ourselves. No.

Yeah, sounds like someone doesn't understand lifetimes and RAII. Even in modern C++ the number of times you have to actually think about memory management instead of lifetimes is basically zero unless you have to work with old libraries.

I guess something like Android Oboe, macOS DriverKit, Windows Runtime C++ Template Library, or C++/WinRT could be considered old libraries then.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#107

Post by Thaxll »

tptacek wrote: post #39

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were usin…

It's a question I ask often in interview, how do you upload a 5GB file over the network with only 1MB of memory.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#108

Post by pjmlp »

tsimionescu wrote: post #94
bluGill wrote: post #78

Earlier quoted context omitted.

With modern C++ your memory checklist is two steps: put it on the stack, put it in a unique_ptr on the stack. There are more steps after that, but you almost never get to them and wouldn't remember them if you discovered the need for them (which is okay because you never get there).

Your checklist is only covering the simplest case, direct ownership of small data structures. I'm not going to put a large array on the stack. I'm not going to pass unique_ptr (exclusive ownership) of every resource I allocate to every caller. I still need to decide between passing a copy, a unique_ptr, a reference, or a shared_ptr. When I design a data structure with interior pointers, I need to define some ownershi…

Not really irrelevant when the said GC language also does value types, e.g.

   // C#
   Span buffer = stackalloc byte[1024];

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#109

Post by UncleEntity »

bluGill wrote: post #78
tsimionescu wrote: post #37

Earlier quoted context omitted.

But thinking about lifetimes and RAII is 90% of memory management. Basically whether you write C, C++, or Rust, you have to track ownership the same ways, the only thing that changes is how much the compiler helps you with that. However, if you write your program in Java, Lisp or Haskell, you simply do not care about ownership for memory-only objects, and can structure your program significantly differently. This can…

With modern C++ your memory checklist is two steps: put it on the stack, put it in a unique_ptr on the stack. There are more steps after that, but you almost never get to them and wouldn't remember them if you discovered the need for them (which is okay because you never get there).

> … put it on the stack, put it in a unique_ptr on the stack.

What happens when the stack frame gets destroyed but you kept a reference to the data around somewhere because you needed it for further compilation?

I, for one, am a fan of using the heap when doing the C++ things…

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#110

Post by josephg »

tptacek wrote: post #84
atombender wrote: post #79

Earlier quoted context omitted.

Reflection is also typically needed for anything that needs to be generic over types. For example, if you want to write a function that can traverse or transform a map or slice, where the actual types aren't known at compile time. We have a lot of this in our Go code at the company I work for. I'm really looking forward to generics, which will help us rip out a ton of reflect calls.

That kind of code is generally non-idiomatic in Go. An experienced Go programmer looks at something that is generic over types and does something interesting and instinctively asks "what gives, where are the dead rabbits?". I'm less excited about generics. There's a cognitive cost to them, and the constraint current Go has against writing type-generic code is often very useful, the same way a word count limit is usef…

I’m so conflicted on that point. I’ve been writing a high performance CRDT in rust for the last few months, and I’m leaning heavily on generics. For example, one of my types is a special b-tree for RLE data. (So each entry is a simple range of values). The b-tree is used in about 3-4 different contexts, each time with a different type parameter depending on what I need. Without genetics I’d need to either duplicate my code or do something simpler (and slower). I can imagine the same library in javascript with dynamic types and I think the result would be easier to read. But the resulting executable would run much slower, and the code would be way more error prone. (I couldn’t lean on the compiler to find bugs. Even TS wouldn’t be rich enough.)

Generics definitely make code harder to write and understand. But they can also be load bearing - for compile time error checking, specialisation and optimization. I’m not convinced it’s worth giving that up.

Post reply on HN