Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Taming Go’s memory usage, or how we avoided rewriting our client in Rust

akitasoftware.com

Post reply on HN
Print view View original

181–190 of 231 posts

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#181

Post by kelnos »

marcus_holmes wrote: post #175
silisili wrote: post #38

Earlier quoted context omitted.

Agreed with this 100%. So many posts here over the years of examples of 'how we rewrote from x to y and saw 2000% gains', where x and y are languages. Such examples are 100% meaningless. Rewrites from the ground up -should- always be way faster, since it's all greenfield. If trying to make a language comparison, rewrite the entire thing in both languages!

Not only greenfield, but the problem domain is much better understood. A lot of architecture choices are made in the early days of a project when the problem isn't sufficiently understood to make the choice correctly. I'm a huge fan of writing the first version of anything as an problem-exploration prototype, intended to be discarded and rewritten. As Fred Brooks said, "you're going to rewrite anyway, you might as we…

In my experience, the prototype never gets thrown away when it should be, and sometimes it's never thrown away at all. It just gets extended, poorly, until development grinds to a halt because you can no longer add features or fix bugs without creating new bugs.

Then you either a) stop what you're doing and spend many months rewriting, or b) spin up a parallel team that does the rewrite, while the old team maintains the old code and does their best to add the most critical features and fix the most critical bugs without breaking anything else in the process.

Neither approach is good. (a) means you'll probably lose customers due to lack of progress on their pet issues. (b) means your development costs have doubled, and you have a team full of people who are demotivated and demoralized because they know they're working on something that's soon destined for the junk heap.

I usually build the first version expecting that it will live on for quite a long time (and sometimes/often be the only version), and build with an eye toward ease of refactor and even ease of rearchitecting. Yes, it's slower than building a prototype-quality product, and yes, sometimes product managers complain that the extra time needed will blow a market opportunity. Those PMs are usually wrong, and even if they are potentially right, building the prototype always takes longer than expected, so the PMs end up fretting over time-to-market anyway.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#182

Post by shakow »

Cthulhu_ wrote: post #179
silisili wrote: post #141

Earlier quoted context omitted.

> They stopped compiling regexps on the fly and moved the regexps to package variables. (I actually don't know if this was a significant win; there might just be the three big wins.) Anecdotally, this could be a huge win, depending on how often it's called. A guy I was working with, new to Go, was writing a router config parser and asked why it was so slow. The first thing I did was moved regexp.Compile from a hot pa…

I think it's easy to assume that in this case Go's regex library would keep an internal cache of expressions, using the expression string as a map key. But on the other, I can see why they haven't implemented it, because it obscures memory usage from direct control of the author. It would probably be a good idea to add performance hints like 'prefer to put static regular expressions in a package variable' in a linter…

> I think it's easy to assume that in this case Go's regex library would keep an internal cache of expressions

IMHO, the stdlib doing implicit memoization is a catastrophe waiting to happen.

I think that handling regexps and caching functions are two composable and orthogonal features that should be handled by two packages/libs/... .

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#183

Post by shantnutiwari »

jgrant27 wrote: post #111

After using Rust for a few years professionally it's my take that people that really want to use it haven't had much experience with it on real world projects. It just doesn't live up to the hype that surrounds it. The memory and CPU savings are negligible between Go and Rust in practice no matter what people might claim in theory. However, the side effects of making your team less productive by using Rust is a much…

>> Simple is better. Stay with Go.

Ive been feeling the same, but as someone who just played with Go/Rust (and never professionally), it's nice to hear that professionals feel the same.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#184

Post by pjmlp »

jerf wrote: post #47
tptacek wrote: post #41

Earlier quoted context omitted.

I think the Rust and Go stories with buffers vs. readers is pretty comparable. They both have good support for readers, and to-good support for reading whole messages into slices or Vec 's.

Good to hear. I hope it's something all new languages have going forward, because like I mentioned in my extended post it's almost all about setting the tone correctly early in the standard library & culture, rather than any sort of "language feature" Go had. As mostly-a-network engineer it's a major pet peeve of mine when I have to step back into some environment where everything works with strings. I can just feel…

You mean just like XML-RPC and JSON-RPC (sorry REST), work?

Because the best way to contribute to global warming is to waste CPU cycles serializing and deserializing data structures into XML and JSON, and parsing them as well.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#185

Post by _ph_ »

Cthulhu_ wrote: post #179
silisili wrote: post #141

Earlier quoted context omitted.

> They stopped compiling regexps on the fly and moved the regexps to package variables. (I actually don't know if this was a significant win; there might just be the three big wins.) Anecdotally, this could be a huge win, depending on how often it's called. A guy I was working with, new to Go, was writing a router config parser and asked why it was so slow. The first thing I did was moved regexp.Compile from a hot pa…

I think it's easy to assume that in this case Go's regex library would keep an internal cache of expressions, using the expression string as a map key. But on the other, I can see why they haven't implemented it, because it obscures memory usage from direct control of the author. It would probably be a good idea to add performance hints like 'prefer to put static regular expressions in a package variable' in a linter…

Actually I would expect any package not to silently cache things until explicitly specified. This otherwise creates an unbounded memory leak.

Moving static (at least as much it concerns the loop) expressions out of a loop is one of the most fundamental optimizations a programmer should do when writing code.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#186

Post by pjmlp »

rossmohax wrote: post #106

Every article on Go allocations can benefit from a heap escape analysis section. I was hoping to find one here, but no luck. Stack allocation is a powerfull technique to reduce GC times.

Agreed, many put all GC languages on the same bag without understanding that several of them (including Go) do provide C like features.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#187

Post by _ph_ »

Thaxll wrote: post #107
tptacek wrote: post #39

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were usin…

It's a question I ask often in interview, how do you upload a 5GB file over the network with only 1MB of memory.

I am even not sure what this question is aiming at - I hope you are phrasing it more detailed than put here, or it would fit in those posts about the problems with interview questions :).

Assuming the file is on a disk and the 1 MB refers to the system memory - like you do with any potentially unbound data, you read and write it in chunks. Reading in data of any kind in whole is only reasonable, if you can clearly set an upper bound for its size.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#188

Post by jhgb »

steveklabnik wrote: post #5
notamy wrote: post #2

Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time…

Yes, Rust kinda doesn't fit super cleanly into a very black/white binary here. It is automatic in the sense that you do not generally call malloc/free. The compiler handles this for you. At the same time, you have a lot more control than you do in a language with a GC, and so to some people, it feels more manual. It's also like, a perception thing in some sense. Imagine someone writes some code. They get a compiler e…

> At the same time, you have a lot more control than you do in a language with a GC

Are there some examples of that?

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#189

Post by NegatioN »

This might be more fit for StackOverflow, but I have a related question.

I have a Go application that runs in Kubernetes, where memory usage steadily increases until it's at around 90% of the cgroup limit, where it seems to stabilize. As far as I can tell, Go GC uses the container memory limits to navigate it's total memory usage (this might be the fault of the OS not reclaiming what Go has already freed(?)).

However, my issue is that in this app, I also call out to cGo, and do manual memory allocations in C++ every 10-30minutes. This works well, except when the container is stabilized at a high memory usage, and my manual allocation brings it over the limit, thus forcing kubernetes to terminate it. (These allocations should as far as I know not be leaking. For a short while, I have two large objects allocated, and 99.9% of the time it's only one)

So, what I'd ideally want is to be able to specify a target heap size for GoGC, and then have a known overhead for the manual allocation. But as far as I'm aware, this isn't possible (?)

Does anyone have any experience with something like this, or see any obvious avenues to pursue to solve the termination issue?

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#190

Post by qaq »

qaq wrote: post #137
Thaxll wrote: post #107

Earlier quoted context omitted.

It's a question I ask often in interview, how do you upload a 5GB file over the network with only 1MB of memory.

hello world in Go is 1.9 MB

I am not saying it's bad given what go runtime can do
Post reply on HN