Taming Go’s memory usage, or how we avoided rewriting our client in Rust • Hacker News Board

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#191

Post by drogus » Wed, Sep 22, 2021, 10:42 AM UTC

jgrant27 wrote: post #111

After using Rust for a few years professionally it's my take that people that really want to use it haven't had much experience with it on real world projects. It just doesn't live up to the hype that surrounds it. The memory and CPU savings are negligible between Go and Rust in practice no matter what people might claim in theory. However, the side effects of making your team less productive by using Rust is a much…

My experience differs quite a bit. I did a bit of production code in Go and a bit of Rust as a hobby + one production Rust service. I guess it might depend on the kind of problems that you work on, but for the most part I don't think that my Rust code is so much different than Go. Definitely more concise. I admit there are times when I have to spend more time to think about how to implement a certain thing, but honestly, if you don't need raw performance you almost always can get away with one of the smart pointers and cloning (or just cloning?). So I don't feel that I'm much slower writing Rust and I'm happy to have more compile type checks.

I don't think that my experience is something isolated, either, here is for example a quote from one of Microsoft employees:

> "For the first week or so, we lost much of our time to learning how borrows worked. After about two weeks, we were back up to 50% efficiency compared to us writing in Go. After a month, we all were comfortable enough that we were back up to full efficiency (in terms of how much code we could write)," writes Thomas.

> "However, we noticed that we gained productivity in the sense that we didn't spend as much time manually checking specific conditions, like null pointers, or not having to debug as many problems."

https://www.zdnet.com/article/microsoft-why-we-used-programm...

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#192

Post by drogus » Wed, Sep 22, 2021, 10:51 AM UTC

ferdowsi wrote: post #140

jgrant27 wrote: post #111

After using Rust for a few years professionally it's my take that people that really want to use it haven't had much experience with it on real world projects. It just doesn't live up to the hype that surrounds it. The memory and CPU savings are negligible between Go and Rust in practice no matter what people might claim in theory. However, the side effects of making your team less productive by using Rust is a much…

Agreed. My organization has been a great testing ground for comparing Go vs Rust service development. The teams that spun up web services in Rust have almost uniformly have had poor experiences. In addition to Rust's steep learning curve, the relatively feature-poor standard library (you have to pull in a third party package to create a SHA256!), and instability of best practices/tools around service writing, in one…

Just out of curiosity - what kind of service was it? My experience with web services (API and websockets) has been great with Rust and actix, so I'm curious if it might be a difference of the work that needed to be done.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#193

Post by drogus » Wed, Sep 22, 2021, 10:57 AM UTC

zarzavat wrote: post #159

roca wrote: post #135

Earlier quoted context omitted.

> the side effects of making your team less productive by using Rust is a much higher price to pay than just running you Go service on more powerful hardware. This entirely depends on the ratio of development effort to deployed instances. At one end of the spectrum, lots of developers work for years on a system which is only deployed on one machine; obviously you optimize for developer effort and buy a single massive…

My problem with Rust: I'm sure that if I used it as my primary language for a couple of years, I would be able to claw the productivity loss back. But I can't find any reason to justify using it at my current productivity level. There is a vicious cycle: few projects use Rust because the productivity hit is large, and programmers do not get enough experience using Rust because few projects use it.

I don't think that 5 years is needed to feel productive. I started Rust a few years ago, but I dropped it due to lack of time and I remember that I had a really hard time with some of the stuff (most notably futures between async/await). I got back in 2019 and I wrote maybe two small projects (under 300 lines of code each I think) and I read quite a lot. After that I got to implement a production web service and also mentor/teach two people. It went very smoothly and for the most part there were no major blockers.

Obviously it all depends on a lot of stuff, but I think that for most people a few weeks to month of writing Rust at work (meaning full time, not like an hour in the evening here or there) should be enough to feel decently productive.

Another thing is that if you've tried Rust long time ago check it out again. I think that both the language and the ecosystem changed a lot in the recent years, it's hard to compare how easy it is to do Rust now vs 2016 or 2017 when I first tried it.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#194

Post by drogus » Wed, Sep 22, 2021, 11:05 AM UTC

shantnutiwari wrote: post #183

jgrant27 wrote: post #111

After using Rust for a few years professionally it's my take that people that really want to use it haven't had much experience with it on real world projects. It just doesn't live up to the hype that surrounds it. The memory and CPU savings are negligible between Go and Rust in practice no matter what people might claim in theory. However, the side effects of making your team less productive by using Rust is a much…

>> Simple is better. Stay with Go. Ive been feeling the same, but as someone who just played with Go/Rust (and never professionally), it's nice to hear that professionals feel the same.

I mean, I'm a professional and I'd say "it depends" (as always), but for most of the stuff that I do I would choose Rust, especially if I care about maximum reliability. Go is statically typed, but I've had situations when there was a runtime exception in Go cause of a mistake that wasn't caught by tests nor code review. In Rust you almost never see runtime exceptions, especially with good linting rules. And thanks to no data races I feel so much more confident writing concurrent code.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#195

Post by drogus » Wed, Sep 22, 2021, 11:13 AM UTC

rapsey wrote: post #170

jgrant27 wrote: post #111

After using Rust for a few years professionally it's my take that people that really want to use it haven't had much experience with it on real world projects. It just doesn't live up to the hype that surrounds it. The memory and CPU savings are negligible between Go and Rust in practice no matter what people might claim in theory. However, the side effects of making your team less productive by using Rust is a much…

This is highly project specific. Go is not suitable for everything. Rust is designed as a C++ replacement not a language for writing backends. Even though a whole lot of effort was put into this space. Go is very good at writing backends, Rust is very good at replacing C++. Everything else the waters get much muddier.

Obviously this is a personal preference, but I prefer Rust for web services. And so I have a question - do you have experience in writing web services with Go and/or Rust? I'm often wondering what do people miss when writing Rust based web services.

Recently I even gave a shot to a todo-backend[1] implementation in Rust[2] and it honestly doesn't look that different from the Go versions.

Granted the todo-backend spec is very very simple. I would prefer to also include stuff like authentication/authorization and maybe even multi tenancy to compare better. But when I'm writing this kind of Rust code I'm often wondering - what makes Rust so unergonomic for other people?

  1. https://todobackend.com/
  2. https://github.com/drogus/todo-backend/blob/main/src/main.rs

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#196

Post by krageon » Wed, Sep 22, 2021, 11:14 AM UTC

Thaxll wrote: post #107

tptacek wrote: post #39

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were usin…

It's a question I ask often in interview, how do you upload a 5GB file over the network with only 1MB of memory.

To such a vague question, the only answer can be "less than 1MB at a time". Not terribly useful.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#197

Post by aba_cz » Wed, Sep 22, 2021, 11:22 AM UTC

silisili wrote: post #141

tptacek wrote: post #39

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were usin…

> They stopped compiling regexps on the fly and moved the regexps to package variables. (I actually don't know if this was a significant win; there might just be the three big wins.) Anecdotally, this could be a huge win, depending on how often it's called. A guy I was working with, new to Go, was writing a router config parser and asked why it was so slow. The first thing I did was moved regexp.Compile from a hot pa…

Spring (boot) works exactly the same. We once found that 30% of CPU time is spent parsing path regexes in Controllers somewhere deep inside the Spring. We had rewritten 1500 endpoints to hardcoded paths and it fixed CPU usage.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#198

Post by rapsey » Wed, Sep 22, 2021, 11:42 AM UTC

drogus wrote: post #195

rapsey wrote: post #170

Earlier quoted context omitted.

This is highly project specific. Go is not suitable for everything. Rust is designed as a C++ replacement not a language for writing backends. Even though a whole lot of effort was put into this space. Go is very good at writing backends, Rust is very good at replacing C++. Everything else the waters get much muddier.

Obviously this is a personal preference, but I prefer Rust for web services. And so I have a question - do you have experience in writing web services with Go and/or Rust? I'm often wondering what do people miss when writing Rust based web services. Recently I even gave a shot to a todo-backend[1] implementation in Rust[2] and it honestly doesn't look that different from the Go versions. Granted the todo-backend spec…

Rust async is not as simple to use, the ecosystem is much smaller and segmented across async-std and tokio.

A good backend stack requires a rich ecosystem of various connectors to databases, cloud services, payment services, frontend stuff like server side rendering, graphql etc.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#199

Post by _ph_ » Wed, Sep 22, 2021, 11:51 AM UTC

typical182 wrote: post #40

Very nice write up. Go’s focus on simplicity means that there is only a single parameter, SetGCPercent, which controls how much larger the heap is than the live objects within it . FWIW, there is a new proposal from a member of the core Go team to add a second GC knob in the form of a soft limit on total memory: https://github.com/golang/proposal/blob/master/design/48409-... It includes some provisions to make sure t…

Hmm, I wonder whether a better alternative might be to be able to set a minimum memory size to use. It is a bit annoying to start a go program when you exactly know you are going to need 1G of memory and after the first 1M allocated it tries to GC before growing the heap. If you could set a minimum memory size, then you could get away with a very low value for GOGC to limit the space overhead beyond your set memory size.

Re: Taming Go’s memory usage, or how we avoided rewriting our client in Rust

#200

Post by anothernewdude » Wed, Sep 22, 2021, 12:02 PM UTC

angelzen wrote: post #16

Spartan-S63 wrote: post #3

Earlier quoted context omitted.

I feel that this is one of those common misconceptions about Rust. Rust's memory management is nothing like C or non-modern C++'s with malloc/free or new/delete. Rust uses modern-C++'s RAII model, typically, to allocate memory. The compiler is smart enough to know when to call drop() (which is essentially free/delete, but with the possibility of additional behavior). You can also call drop() yourself. What I think pe…

Tangentially, I did a bit of Rust work recently. I was sadly unable to find a concise credible answer to a rather elementary best-practices question: How does ownership interact with nested datastructures? Is it possible to build a heap tree without Boxing every node explicitly?

Of course. Arena allocation comes to mind.