Generics can make your Go code slower • Hacker News Board

Re: Generics can make your Go code slower

#191

Post by bcrosby95 » Wed, Mar 30, 2022, 8:19 PM UTC

emtel wrote: post #182

titzer wrote: post #81

Earlier quoted context omitted.

You really haven't given any supporting information for your argument other than a vague feeling that GC is somehow bad. In fact you just pointed out many counterexamples to your own argument, so I'm not sure what to take away. I've seen this sentiment a lot, and I never see specifics. "GC is bad for systems language" is an unsupported, tribalist, firmly-held belief that is unsupported by hard data. On the other hand…

My argument against GC (and which applies similary to JIT-basd runtimes) is that the problems caused by GC pauses have non-local causes. If a piece of code ran slowly because of a GC pause, the cause of the pause is in some sense _the entire rest of the system_. You can't fix the problem with a localized change. Programs in un-managed languages can be slow too, and excessive use of malloc() is a frequent culprit. But…

I don't want to try to bring up an exception that disproves your rule, but what about something like BEAM, where it has per-process (process = lightweight thread) heaps and GC.

Re: Generics can make your Go code slower

#192

Post by nikki93 » Wed, Mar 30, 2022, 8:20 PM UTC

pphysch wrote: post #29

My first use of Go generics has been for a concurrent "ECS" game engine. In this case, the gains are pretty obvious. I think. I get to write one set of generic methods and data structures that operate over arbitrary "Component" structs, and I can allocate all my components of a particular type contiguously on the heap, then iterate over them with arbitrary, type-safe functions. I can't fathom that doing this via a Co…

Sweet! I've been using it for the same. Example game project (did it for a game jam): https://github.com/nikki93/raylib-5k -- in this case the Go gets transpiled to C++ and runs as WebAssembly too. Readme includes a link to play the game in the browser. game.gx.go and behaviors.gx.go kind of show the ECS style.

It's worked with 60fps performance on a naive N^2 collision algo over about 4200 entities -- but also I tend to use a broadphase for collisions in actual games (there's an "externs" system to call to other C/C++ and I use Chipmunk's broadphase).

Re: Generics can make your Go code slower

#193

Post by pjmlp » Wed, Mar 30, 2022, 8:21 PM UTC

ptsneves wrote: post #159

cb321 wrote: post #124

Earlier quoted context omitted.

>until recently Actually, for a long time (almost 20 years, I think), the gcc subsystem gcj let you do AOT compilation of Java to ELF binaries. [1] I think they had to be dynamically linked, but only to a few shared objects (unless you pulled in a lot via native dependencies, but that's kind of "on you"). I don't recall any restrictions on how to use generated code, anymore than gcc-generated object code. So, I don't…

The interesting thing I always heard in the Java world is that AOT was actually a drawback as the virtual machine allowed for just in time optimizations according to hotspots. Actually if I remember correctly the word hotspot itself was even used as a tech trademark. I was always a bit skeptical but given i was never much into Java i just assumed my skepticism was out of ignorance. Now with what I know about profile…

Sun was ideologically against AOT, as all commercial vendors always had some form of either AOT or JIT caches.

In fact, the JIT caches in OpenJDK come from Oracle/Bea's J/Rockit, while IBM's OpenJ9 AOT compilation is from WebSphere Real Time JVM implementation.

Re: Generics can make your Go code slower

#194

Post by initplus » Wed, Mar 30, 2022, 8:21 PM UTC

no_circuit wrote: post #129

reikonomusha wrote: post #64

Earlier quoted context omitted.

I have a slightly contrary opinion. Systems software is a very large umbrella, and much under that umbrella is not encumbered by a garbage collector whatsoever. (To add insult to injury, the term's definition isn't even broadly agreed upon, similar to the definition of a "high-level language".) Yes, there are some systems applications where a GC can be a hindrance in practice, but these days I'm not even sure it's a…

The article is from a database company, so I'll assume that approximates the scope. My scope for the GC discussion would include other parts that could be considered similar software: cluster-control plane (Kubernetes), other databases, and possibly the first level of API services to implement a service like an internal users/profiles or auth endpoints. The tricky thing is GC works most of the time, but if you are wo…

It's impossible to spend any time tuning Go's GC parameters as they intentionally do not provide any.

Go's GC is optimized for latency, it doesn't see the same kind of 1% peak latency issues you get in languages with a long tail of high latency pauses.

Also consider API design - Java API (both in standard & third party libs) tend to be on the verbose side and build complex structures out of many nested objects. Most Go applications will have less nesting depth so it's inherently an easier GC problem.

System designs that rely on allocating a huge amount of memory to a single process exist in a weird space - big enough that perf is really important, but small enough that single-process is still a viable design. Building massive monoliths that allocate hundreds of Gb's at peak load just doesn't seem "in vogue" anymore.

If you are building a distributed system keeping any individual processes peak allocation to a reasonable size is almost automatic.

Re: Generics can make your Go code slower

#195

Post by fluoridation » Wed, Mar 30, 2022, 8:23 PM UTC

ptsneves wrote: post #159

cb321 wrote: post #124

Earlier quoted context omitted.

>until recently Actually, for a long time (almost 20 years, I think), the gcc subsystem gcj let you do AOT compilation of Java to ELF binaries. [1] I think they had to be dynamically linked, but only to a few shared objects (unless you pulled in a lot via native dependencies, but that's kind of "on you"). I don't recall any restrictions on how to use generated code, anymore than gcc-generated object code. So, I don't…

The interesting thing I always heard in the Java world is that AOT was actually a drawback as the virtual machine allowed for just in time optimizations according to hotspots. Actually if I remember correctly the word hotspot itself was even used as a tech trademark. I was always a bit skeptical but given i was never much into Java i just assumed my skepticism was out of ignorance. Now with what I know about profile…

I've heard the exact opposite. The supposed performance benefits of JIT compared to AOT (profile-guided optimization, run-time uarch-targeted optimization) never really materialized. There's been a lot of research since the late '90s into program transformation and it turned out that actually the most effective optimizations are architecture-independent and too expensive to be performed over and over again at startup or when the program is running. At the same time, deciding when it's worthwhile to reoptimize based on new profiling data turned out to be a much more difficult problem than expected.

So the end result is that while both AOT (GCC, LLVM) and JIT (JVM, CLR) toolchains have been making gradual progress in performance, the JIT toolchains never caught up with the AOT ones as was expected in the '90s.

Re: Generics can make your Go code slower

#196

Post by titzer » Wed, Mar 30, 2022, 8:24 PM UTC

throwaway894345 wrote: post #179

fauigerzigerk wrote: post #153

Earlier quoted context omitted.

>I will assume by "infrastructure code" you mean things like kernels and network stacks. That, and things like database systems, libraries that are used in a lot of other software or language runtimes for higher level languages. >The actual heap footprint of the Linux kernel is pretty small And what would that footprint be if the kernel was written in Java or Go? What would the performance of all those device drivers…

> You can of course write memory efficient code in GC languages by manually managing a bunch of buffers. But I have seen and written quite a bit of that sort of code. It's horribly unsafe and horribly unproductive to write. Go uses buffers pretty idiomatically and they don't seem unsafe or unproductive. Maybe I'm not following your meaning? > If safety was ever good enough reason to use GC languages for systems/infra…

> It's also not hard to conceive of a language inspired by Go that is more aggressively optimized (for example, fully monomorphized generics, per this article or with a more sophisticated garbage collector).

Standard ML as implemented by MLton uses full monomorphization and a host of advanced functional optimizations. That code can be blazingly fast. MLton does take a long time to compile, though.

I've been working on a systems language that is GC'd and also does full monomorphization--Virgil. I am fairly miserly with memory allocations in my style and the compiler manages to compile itself (50KLOC) at full optimization in 300ms and ~200MB of memory, not performing a single GC. My GC is really dumb and does a Cheney-style semispace copy, so it has horrible pauses. Even so, GC is invisible so the algorithm could be swapped out at any time.

For an OS kernel, I think a GC would have to be pretty sophisticated (concurrent, highly-parallel, on-the-fly), but I think this is a problem I would love to be working on, rather than debugging the 19000th use-after-free bug.

Go's GC is very sophisticated, with very low pause times. It trades memory for those low pause times and can suffer from fragmentation because it doesn't compact memory. Concurrent copying is still a hard problem.

Again, a problem I'd rather we had than the world melting down because we chose performance rather than security.

Re: Generics can make your Go code slower

#197

Post by bborud » Wed, Mar 30, 2022, 8:27 PM UTC

nvarsj wrote: post #46

I'd argue that golang is inherently not a systems language, with its mandatory GC managed memory. I think it's a poor choice for anything performance or memory sensitive, especially a database. I know people would disagree (hence all the DBs written in golang these days, and Java before it), but I think C/C++/Rust/D are all superior for that kind of application. All of which is to say, I don't think it matters. Use t…

Let’s start at the beginning. What is a «systems language» and for what is it typically used?

Re: Generics can make your Go code slower

#198

Post by SomeCallMeTim » Wed, Mar 30, 2022, 8:28 PM UTC

chakkepolja wrote: post #109

SomeCallMeTim wrote: post #51

Earlier quoted context omitted.

In C++, generics (templates) are zero-cost abstractions. So no, generics do not de facto make code slower.

We don't know of a way to implement generic types without (vtable dispatch + boxing) cost AND without monomorphization cost. Some languages do former, some latter, some combination of 2. Monomorphization: * code bloat * slow compiles * debug builds may be slow (esp c++) Dynamic dispatch & boxing (Usually both are needed): * not zero cost Pick your poison

"Zero-cost" in that context refers to runtime performance. It always refers to runtime performance.

And code bloat, as I've said elsewhere, is vastly overblown as a problem. Another commenter pointed out that link-time optimization removes most of the bloat. The rest is customized code that's optimized per-instantiation.

Slow compiles are an issue with C++ templates. They're literally a Turing-complete code-generation language of their own, and they can perform complex calculations at compile time, so yes, they tend to make compiles take longer when you're using them extensively. But the point I was making was about runtime performance. That's why C++ compilers often perform incremental compilation, which can limit the development time cost.

Debug builds can simply be slow in C++ with or without templates. C++ templates really don't affect debug build runtime performance in any material fashion; writing the code out customized for each given type should have identical performance to the template-generated version of the code, unless there's some obscure corner case I'm not considering.

Re: Generics can make your Go code slower

#199

Post by MaulingMonkey » Wed, Mar 30, 2022, 8:29 PM UTC

throwaway894345 wrote: post #174

titzer wrote: post #81

Earlier quoted context omitted.

You really haven't given any supporting information for your argument other than a vague feeling that GC is somehow bad. In fact you just pointed out many counterexamples to your own argument, so I'm not sure what to take away. I've seen this sentiment a lot, and I never see specifics. "GC is bad for systems language" is an unsupported, tribalist, firmly-held belief that is unsupported by hard data. On the other hand…

Virtually invariably, "GC is bad" assumes (1) lots of garbage (2) long pause times. Go has idiomatic value types (so it generates much less garbage) and a low-latency garbage collector. People who argue against GC are almost always arguing against some Java GC circa 2005.

This is the no true scottsman argument. I mean, no true modern GC. And it's bullshit. Let's be topical and pick on Go since that's the language in the title:

https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i...

30% of CPU spent on GC, individual GC pauses already in the milliseconds, despite a tiny half-gig heap in 2019. For gamedev, a single millisecond in the wrong place can be enough to miss vsync and have unacceptable framerate stutter. In the right place, it's "merely" 10% of your entire frame budget, for VR-friendly, nausea-avoiding framerates near 100fps. Or perhaps 90% if "single digit milliseconds" might include 9ms.

Meanwhile, the last professional project I worked on had 100ms pauses every 30 seconds because we were experimenting with duktape, which is still seeing active commits. Closer to a 32GB heap for that project, but most of that was textures. Explicit allocation would at least show where the problematic garbage/churn was in any profiler, but garbage collection meant a single opaque codepath for all garbage deallocation... without even the benefit of explicit static types to narrow down the problem.

Re: Generics can make your Go code slower

#200

Post by komuW » Wed, Mar 30, 2022, 8:30 PM UTC

masklinn wrote: post #180

komuW wrote: post #118

Go does has some form of monomorphization implemented in Go1.18; it is just behind a feature flag(compiler flags). Look at the assembly difference between this two examples: 1. https://godbolt.org/z/7r84jd7Ya (without monomorphization) 2. https://godbolt.org/z/5Ecr133dz (with monomorphization) If you don't want to use godbolt, run the command `go tool compile '-d=unified=1' -p . -S main.go` I guess that the flag is n…

FWIW you can have two different compilers (and outputs) for the same input: https://godbolt.org/z/bb1oG9TbP in the compiler pane just click "add new" and "clone compiler" (you can actually drag that button to immediately open the pane in the right layout instead of having your layout move to vertical thirds and needing to move the pane afterwards). Learned that watching one of Matt's cppcon talks (A+, would do again)…

thanks!