Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Taming Go’s memory usage, or how we avoided rewriting our client in Rust

akitasoftware.com

Post reply on HN
Print view View original

121–130 of 231 posts

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#121

Post by angelzen »

jgrant27 wrote: post #111

After using Rust for a few years professionally it's my take that people that really want to use it haven't had much experience with it on real world projects. It just doesn't live up to the hype that surrounds it. The memory and CPU savings are negligible between Go and Rust in practice no matter what people might claim in theory. However, the side effects of making your team less productive by using Rust is a much…

Explicitly managed memory is useful for handling buffers. Everything else is peanuts anyways and could use a GC for ergonomics reasons. That being said, some really prefer the ergonomics of working with Result and combinators compared with the endless litany "x, err = foo(); if err !== null". IMHO there is still room for significant progress in this space, neither Rust nor Go have hit the sweetspot yet.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#122

Post by IshKebab »

jgrant27 wrote: post #111

After using Rust for a few years professionally it's my take that people that really want to use it haven't had much experience with it on real world projects. It just doesn't live up to the hype that surrounds it. The memory and CPU savings are negligible between Go and Rust in practice no matter what people might claim in theory. However, the side effects of making your team less productive by using Rust is a much…

Why do you say "less productive with Rust"? In my experience I'm more productive with Rust because it's very strong type system catches so many bugs.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#123

Post by pgwhalen »

tptacek wrote: post #117
pgwhalen wrote: post #115

Earlier quoted context omitted.

This argument is getting a little tiresome though, isn't it? It isn't simply enough to call something "non-idiomatic" to gloss over a deficiency. There's a cognitive cost to all language features, but most other general purpose statically typed programming languages seem to have come to the conclusion that the benefit outweighs the cost for some form of generics. I am by no means a Go basher, it is one of my favorite…

I could have written this more clearly. The fact that things that are generic over types are non-idiomatic today in Go has nothing to do with whether the upcoming generics feature is good or bad. They're unrelated arguments. The latter argument is subjective and you might easily disagree. The former argument, about experienced Go programmers being wary when an API is generic over types, is pretty close to an objectiv…

That's a fair point. Knowing not to try to write generic code (since you don't have the tools) is the sign of an experienced Go programmer.

That being said, I'm curious how much kubernetes (a large, famous, Go codebase) still has code that does this. I used to read that it used a ton of interface{} and type assertion, but maybe that narrative is out of date (or never really true). I was never too familiar with the codebase myself.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#124

Post by josephg »

silisili wrote: post #38
xondono wrote: post #15

Earlier quoted context omitted.

> Buried in here are great examples of why rewrites don’t help That has not been my experience. Rewrites do sometimes help, because in a lot of codebases there’s too many “pet” modules or badly designed frozen interfaces. Rewrites can help in those situations, because there’s no sacred cows anymore. The issue is that a lot of people do rewrites as translations, without touching structures.

Agreed with this 100%. So many posts here over the years of examples of 'how we rewrote from x to y and saw 2000% gains', where x and y are languages. Such examples are 100% meaningless. Rewrites from the ground up -should- always be way faster, since it's all greenfield. If trying to make a language comparison, rewrite the entire thing in both languages!

Yes absolutely. I wrote an article a couple months ago which was trending here where I got a 5000x performance improvement over an existing system. One of the changes I made was moving to rust, and some people seemed to think the takeaway was “rewriting the code in rust made it 5000x faster”. It wasn’t that. Automerge already had a rust version of their code which ran a benchmark in 5 minutes. Yjs does the same benchmark in less than 1 second in javascript.

Yjs is so fast because it makes better choices with its data structures. A recent PR in automerge-rs brought the same 5 minute test down to 2 seconds by changing the data structure it uses.

Rust/C/C++ give you more tools to write high performance code. But if you put everything on the heap with copies everywhere, your code won’t be necessarily any faster than it would in JS / python / ruby. And on the flip side, you can achieve very respectable performance in dynamic languages with a bit of care along the hot path.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#125

Post by jcelerier »

oconnor663 wrote: post #48
jcelerier wrote: post #26

Earlier quoted context omitted.

It kills me that RAII is considered modern c++. It's there since 1983 aha, what do you think fstream and std::vector are if not RAII wrappers over files or memory

I think before the introduction of move semantics in C++11, there were a lot of cases where you needed new and delete to get basic things working. (Moving an fstream around is a relevant example.) So the modern rule of "don't use new and delete in application code" really wasn't practical before that.

No, pretty much everything could be done with swap (like moving an fstream as you say). Sure, it's a bit more cumbersome, but it was still RAII.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#126

Post by laumars »

bluGill wrote: post #69
laumars wrote: post #65

Earlier quoted context omitted.

This is where profiling helps more. Find the weak parts of the code, try to optimise those. If the language proves to be a barrier then you have a justification for a rewrite. All too often people don’t understand how to performance tune software properly and instead blame other things first (eg garbage collection)

Most slow languages make escape to C easy for cases where the language is the issue. Most fast languages make writing a C APIed interface easy, so if the language is your issue just rewrite the parts where that is the problem. Of course eventually you get to the point where enough of the code is in a fast language that writing everything in the fast language to avoid the pain of language interfaces is worth it.

And there’s time when even C isn’t sufficient and a developer needs to resort to inlined assembly. But most of the time the starting language (whatever that might be) is good enough. Even here, the issue wasn’t the language, it was the implementation. And even where the problem is the language, there will always be hot paths that need hardware performant code (be that CPU, memory, or sometimes other devices like disk IO) and there will be other parts in most programs that need to be optimised for developer performance.

Not everyone is writing sqlite or kernel development level software. Most software projects are a trade off of time vs purity.

That all said, backend web development is probably the edge case here. But even there, that’s only true if you’re trying to serve several thousand requests a second on a monolithic site in something like CGI/Perl. Then I’d argue there’s not point fixing any hot paths and just rewrite the entire thing. But even then, there’s still no need to jump straight to C, skipping Go, Java, C#, and countless others.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#127

Post by jhgb »

Thaxll wrote: post #107
tptacek wrote: post #39

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were usin…

It's a question I ask often in interview, how do you upload a 5GB file over the network with only 1MB of memory.

> with only 1MB of memory

Is that total system memory?

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#128

Post by marcos100 »

wibagusto wrote: post #112
maleldil wrote: post #83

Earlier quoted context omitted.

> subtly calling attention That's generous. I'd call it clickbait.

Well consider all the projects titled “blah blah blah… written in Rust” Who gives a shit what it’s written in—what does it do?

People who is interested in rust maybe want to see how it was used.

The author could have just kept "Taming Go's Memory Usage".

Maybe they never considered rewriting in rust. The pros and cons looks like just some random arguments to add rust to the title.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#129

Post by wrs »

xondono wrote: post #15
wrs wrote: post #10

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the langua…

> Buried in here are great examples of why rewrites don’t help That has not been my experience. Rewrites do sometimes help, because in a lot of codebases there’s too many “pet” modules or badly designed frozen interfaces. Rewrites can help in those situations, because there’s no sacred cows anymore. The issue is that a lot of people do rewrites as translations, without touching structures.

Quite true, a rewrite can help if it is also a "rethink". But you don't have to switch languages to get that effect--in fact you'll probably do better if you don't throw a new language/library into the mix.

My point was that, contrary to what is apparently a common impulse, rewriting the same thing in a different language while maintaining the lack of attention to performance considerations that was present in the first version isn't going to help much.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#130

Post by jhgb »

Karrot_Kream wrote: post #113
jrockway wrote: post #103

Earlier quoted context omitted.

Agree strongly here. These are common sources of memory leaks in any language, and it's very likely that rewriting this code in Rust would lead to the exact same problems. (Other cases on HN, like Discord's in-memory cache and Twitch's "memory ballast" thing, are pretty Go specific -- the identical C program wouldn't have those particular bugs. But, the Go developers read these incident reports and do fix the underly…

> it's an intrinsic Law Of The Universe that if data comes in a X bytes per second, and leaves at X-k bytes per second, then eventually you will use all storage space in the Universe for your buffer, This is known as Little's Law. Using Little's Law, you know that if the average time spent in queue is more than the average time it takes for a new entry to be added to the queue, then your queue fills up.

Did Little formulate multiple eponymous laws? Since that does not seem to be the Little's law that I'm familiar with.
Post reply on HN