Generics can make your Go code slower • Hacker News Board

Re: Generics can make your Go code slower

#241

Post by titzer » Wed, Mar 30, 2022, 9:49 PM UTC

ninkendo wrote: post #239

titzer wrote: post #114

Earlier quoted context omitted.

Indeed. What really kills is extremely high allocation rates and extremely high garbage production. I've seen internal numbers from $megacorp that show that trashy C++ programs (high allocation + deallocation rates) look pretty much the same to CPUs as trashy Java programs, but are far worse in terms of memory fragmentation. Trashy C++ programs can end up spending 20+% of their execution time in malloc/free. That's a…

> I will admit that the programming culture is different for many GC'd languages' communities, sometimes encouraging a very trashy programming style, which contributes to the perception that GC itself is the problem For some languages (I’m looking at you, Java), there’s not much of a way to program that doesn't generate a bunch of garbage, because only primitives are treated as value types, and for Objects, heap allo…

> I’m looking at you, Java

> Sometimes it’s appropriate to blame the language.

Oh, I know, I was just being vague to be diplomatic. Java being generally trashy has been one of the major motivators for me to do Virgil. In Java, you can't even parse an integer without allocating memory.

Re: Generics can make your Go code slower

#242

Post by socialdemocrat » Wed, Mar 30, 2022, 9:49 PM UTC

nvarsj wrote: post #46

I'd argue that golang is inherently not a systems language, with its mandatory GC managed memory. I think it's a poor choice for anything performance or memory sensitive, especially a database. I know people would disagree (hence all the DBs written in golang these days, and Java before it), but I think C/C++/Rust/D are all superior for that kind of application. All of which is to say, I don't think it matters. Use t…

I would say Go is a systems programming language. A systems programming language is for creating services used by actual end user applications. That is pretty much what Go is being used for. Who is writing editors or drawing applications in Go? Nobody.

Go does contain many of the things of interest to systems programmers such as pointers and the ability to specify memory layout of data structures. You can make your own secondary allocators. In short it gives you far more fine grained control over how memory is used than something like Java or Python.

https://erik-engheim.medium.com/is-go-a-systems-programming-...

Re: Generics can make your Go code slower

#243

Post by KwisaksHaderach » Wed, Mar 30, 2022, 9:50 PM UTC

RedShift1 wrote: post #173

zachruss92 wrote: post #145

For me Go has replaced Node as my preferred backend language. The reason is because of the power of static binaries, the confidence that the code I write today can still run ten years from now, and the performance. The difference in the code I’m working with is being able to handle 250 req/s in node versus 50,000 req/s in Go without me doing any performance optimizations. From my understanding Go was written with dev…

250 vs. 50000 req/s seems like a too big of a difference to me. Sure Go is faster than Node but Node is no slough either, you might want to dig in some deeper why you only got 250 req/s with Node.

Doesn't sound unrealistic if you have a mix load of IO and raw processing.

Re: Generics can make your Go code slower

#244

Post by [deleted] » Wed, Mar 30, 2022, 9:50 PM UTC

MaulingMonkey wrote: post #233

titzer wrote: post #216

Earlier quoted context omitted.

From your link (which I remember reading at the time): > So by simply reducing GC frequency, we saw close to a ~99% drop in mark assist work, which translated to a~45% improvement in 99th percentile API latency at peak traffic. Did you look at the actual article? (Because it doesn't support your point). They added a 10GB memory ballast to keep the GC pacer from collecting too much. That is just a bad heuristic in the…

> Please, we can keep the temperature on the conversation down a bit by just keeping to facts and leaving out a few of these words. Sure. Let's avoid some of these words too: > unsupported, tribalist, firmly-held belief that is unsupported by hard data. Asking for examples is fine and great, but painting broad strokes of the dissenting camp before they have a chance to respond does nothing to help keep things cool.

[deleted]

Re: Generics can make your Go code slower

#245

Post by erik_seaberg » Wed, Mar 30, 2022, 9:52 PM UTC

initplus wrote: post #194

no_circuit wrote: post #129

Earlier quoted context omitted.

The article is from a database company, so I'll assume that approximates the scope. My scope for the GC discussion would include other parts that could be considered similar software: cluster-control plane (Kubernetes), other databases, and possibly the first level of API services to implement a service like an internal users/profiles or auth endpoints. The tricky thing is GC works most of the time, but if you are wo…

It's impossible to spend any time tuning Go's GC parameters as they intentionally do not provide any. Go's GC is optimized for latency, it doesn't see the same kind of 1% peak latency issues you get in languages with a long tail of high latency pauses. Also consider API design - Java API (both in standard & third party libs) tend to be on the verbose side and build complex structures out of many nested objects. Most…

You tune Go’s GC by rewriting your code. It’s like turning a knob but slower and riskier.

Re: Generics can make your Go code slower

#246

Post by anonymoushn » Wed, Mar 30, 2022, 9:54 PM UTC

kevwil wrote: post #130

Seems obvious; like, did someone expect all the extra abstraction would make Go faster?

Yes? We used code generators to monomorphize our code in like 2015 and it was faster than using interfaces. Generics could reasonably produce the same code we did in 2015, but they don't.

Re: Generics can make your Go code slower

#247

Post by socialdemocrat » Wed, Mar 30, 2022, 9:55 PM UTC

coder543 wrote: post #143

no_circuit wrote: post #129

Earlier quoted context omitted.

The article is from a database company, so I'll assume that approximates the scope. My scope for the GC discussion would include other parts that could be considered similar software: cluster-control plane (Kubernetes), other databases, and possibly the first level of API services to implement a service like an internal users/profiles or auth endpoints. The tricky thing is GC works most of the time, but if you are wo…

Go doesn’t offer a bunch of GC tuning parameters. Really only one parameter, so your concerns about complex GC tuning here seem targeted at some other language like Java. This is a drawback in some cases, since one size never truly fits all, but it dramatically simplifies things for most applications, and the Go GC has been tuned for many years to work well in most places where Go is commonly used. The developers of…

Java _needs_ lots of GC tuning parameters because you have practically no way of tuning the way your memory is used and organized in Java code. In Go you can actually do that. You can decide how data structures are nested, you can take pointers to the inside of a a block of memory. You could make e.g. a secondary allocator, allocating objects from a contiguous block of memory.

Java doesn't allow those things, and thus it must instead give you lots of levers to pull on to tune the GC.

It is just a different strategy of achieving the same thing:

https://itnext.io/go-does-not-need-a-java-style-gc-ac99b8d26...

Re: Generics can make your Go code slower

#248

Post by no_circuit » Wed, Mar 30, 2022, 10:05 PM UTC

coder543 wrote: post #143

no_circuit wrote: post #129

Earlier quoted context omitted.

The article is from a database company, so I'll assume that approximates the scope. My scope for the GC discussion would include other parts that could be considered similar software: cluster-control plane (Kubernetes), other databases, and possibly the first level of API services to implement a service like an internal users/profiles or auth endpoints. The tricky thing is GC works most of the time, but if you are wo…

Go doesn’t offer a bunch of GC tuning parameters. Really only one parameter, so your concerns about complex GC tuning here seem targeted at some other language like Java. This is a drawback in some cases, since one size never truly fits all, but it dramatically simplifies things for most applications, and the Go GC has been tuned for many years to work well in most places where Go is commonly used. The developers of…

Yes, my comments were targeted to Java and Scala. Java has paid the bills for me for many years. I'd use Java for just about anything except for high load infrastructure systems. And if you're in, or want to be in, that situation, then why risk finding out two years later that a GC-enabled app is suboptimal?

I'd guess you'd have no choice if in order to hire developers, you had to choose a language that the people found fun to use.

Re: Generics can make your Go code slower

#249

Post by MaulingMonkey » Wed, Mar 30, 2022, 10:07 PM UTC

titzer wrote: post #232

MaulingMonkey wrote: post #226

Earlier quoted context omitted.

> Did you look at the actual article? (Because it doesn't support your point). I did and it does for the point I intended to derive from said article: >> However, the GC pause times before and after the change were not significantly different. Furthermore, our pause times were on the order of single digit milliseconds, not the 100s of milliseconds improvement we saw at peak load. They were able to improve times via t…

> but those numbers mean you'd want to avoid doing anything at all in a gamedev render thread that could potentially trigger a GC pause, because said GC pause will trigger a vsync miss. You might want to take a look at this: https://queue.acm.org/detail.cfm?id=2977741

I have, it's a decent read - although somewhat incoherent. E.g. they tout the benefits of GCing when idle, then trash the idea of controlling GC:

> Sin two: explicit garbage-collection invocation. JavaScript does not have a Java-style System.gc() API, but some developers would like to have that. Their motivation is proactively to invoke garbage collection during a non-time-critical phase in order to avoid it later when timing is critical. [...]

So, no explicitly GCing when a game knows it's idle. Gah. The worst part is these are entirely fine points... and somewhat coherent in the context of webapps and webpages. But then when one attempts to embed v8 - as one does - and suddenly you the developer are the one that might be attempting to time GCs correctly. At least then you have access to the appropriate native APIs:

* https://v8docs.nodesource.com/node-7.10/d5/dda/classv8_1_1_i... * https://v8docs.nodesource.com/node-7.10/d5/dda/classv8_1_1_i...

A project I worked on had a few points where it had to explicitly call GC multiple times back to back. Intertwined references from C++ -> Squirrel[1] -> C++ -> Squirrel meant the first GC would finalize some C++ objects, which would unroot some Squirrel objects, which would allow some more C++ objects fo be finalized - but only one layer at a time per GC pass.

Without the multiple explicit GC calls between unrooting one level and loading the next, the game had a tendency to "randomly"[2] ~double it's typical memory budget (thanks to uncollected dead objects and the corresponding textures they were keeping alive), crashing OOM in the process - the kind of thing that would fail console certification processes and ruin marketing plans.

[1]: http://squirrel-lang.org/

[2]: quite sensitive to the timing of "natural" allocation-triggered GCs, and what objects might've created what reference cyles etc.

Re: Generics can make your Go code slower

#250

Post by titzer » Wed, Mar 30, 2022, 10:07 PM UTC

fauigerzigerk wrote: post #210

throwaway894345 wrote: post #179

Earlier quoted context omitted.

> You can of course write memory efficient code in GC languages by manually managing a bunch of buffers. But I have seen and written quite a bit of that sort of code. It's horribly unsafe and horribly unproductive to write. Go uses buffers pretty idiomatically and they don't seem unsafe or unproductive. Maybe I'm not following your meaning? > If safety was ever good enough reason to use GC languages for systems/infra…

My argument is about the economics of software development more than about any of the large number of interesting technical details we could debate for a very long time. There are higher level features that cause higher resource consumption. GC is clearly one such feature. No one denies that. So how do we decide where it makes more sense to use these features and where does it make less sense? What I'm saying is that…

I understand your argument, it's been made for decades. Put in a lot of effort to save those resources. But isn't about effort. We put in a lot of effort and still got crap, even worse, security. We put effort into the wrong things!

We've majorly screwed up our priorities. Correctness should be so much higher up the priority list, probably #1, TBH. When it is a high priority, we should be willing to sacrifice performance to actually get it. The correct question is not if we should sacrifice performance, but how much. We didn't even get that right.

But look, I know. Security doesn't sell systems, never has--benchmarks do. The competitive benchmarking marketplace is partly responsible. And there, there's been so much FUD on the subject that I feel we've all been hoodwinked and conned into putting performance at the top to all of our detriment. That was just dumb on our (collective) part.

Let me put it another way. Go back to 1980. Suppose I offered you two choices. Choice A, you get a 1000x improvement in computer performance and memory capacity, but your system software is a major pain in the ass to write and full of security vulnerabilities, to the point where the world suffers hundreds of billions of dollars of lost GDP due to software vulnerabilities. Choice B, you get an 800x improvement in computer performance, a 500x improvement in memory capacity, and two thirds of that GDP loss just doesn't happen. Also, writing that software isn't nearly as much of a pain in the ass.

Which did we choose? Yeah. That's where the disagreement lies.