Generics can make your Go code slower • Hacker News Board

Re: Generics can make your Go code slower

#231

Post by EdwardDiego » Wed, Mar 30, 2022, 9:28 PM UTC

no_circuit wrote: post #129

reikonomusha wrote: post #64

Earlier quoted context omitted.

I have a slightly contrary opinion. Systems software is a very large umbrella, and much under that umbrella is not encumbered by a garbage collector whatsoever. (To add insult to injury, the term's definition isn't even broadly agreed upon, similar to the definition of a "high-level language".) Yes, there are some systems applications where a GC can be a hindrance in practice, but these days I'm not even sure it's a…

The article is from a database company, so I'll assume that approximates the scope. My scope for the GC discussion would include other parts that could be considered similar software: cluster-control plane (Kubernetes), other databases, and possibly the first level of API services to implement a service like an internal users/profiles or auth endpoints. The tricky thing is GC works most of the time, but if you are wo…

> A good portion of production outages are likely related to cascading failures due to too long GC pauses, and a good portion of developer time is spent testing and tuning GC parameters

After 14 years in JVM dev in areas where latency and reliability are business critical, I disagree.

Yes, excessive GC stop the world pauses can cause latency spikes, and excessive GC time is bad, and yes, when a new GC algorithm is released that you think might offer improvements, you test it thoroughly to determine if it's better or worse for your workload.

But a "good portion" of outages and developer time?

Nope. Most outages occur for the same old boring reasons - someone smashed the DB with an update that hits a pathological case and deadlocks processes using the same table, a DC caught fire, someone committed code with a very bad logical bug, someone considered a guru heard that gRPC was cool and used it without adequate code review and didn't understand that gRPC's load balancing defaults to pick first, etc. etc.

The outages caused by GC were very very few.

Outages caused by screw-ups or lack of understanding of subtleties of a piece of tech, as common as they are in every other field of development.

Then there's the question of what outages GCed languages _don't_ suffer.

I've never had to debug corrupted memory, or how a use after free bug let people exfiltrate data.

Re: Generics can make your Go code slower

#232

Post by titzer » Wed, Mar 30, 2022, 9:29 PM UTC

MaulingMonkey wrote: post #226

titzer wrote: post #216

Earlier quoted context omitted.

From your link (which I remember reading at the time): > So by simply reducing GC frequency, we saw close to a ~99% drop in mark assist work, which translated to a~45% improvement in 99th percentile API latency at peak traffic. Did you look at the actual article? (Because it doesn't support your point). They added a 10GB memory ballast to keep the GC pacer from collecting too much. That is just a bad heuristic in the…

> Did you look at the actual article? (Because it doesn't support your point). I did and it does for the point I intended to derive from said article: >> However, the GC pause times before and after the change were not significantly different. Furthermore, our pause times were on the order of single digit milliseconds, not the 100s of milliseconds improvement we saw at peak load. They were able to improve times via t…

> but those numbers mean you'd want to avoid doing anything at all in a gamedev render thread that could potentially trigger a GC pause, because said GC pause will trigger a vsync miss.

You might want to take a look at this:

https://queue.acm.org/detail.cfm?id=2977741

Re: Generics can make your Go code slower

#233

Post by MaulingMonkey » Wed, Mar 30, 2022, 9:34 PM UTC

titzer wrote: post #216

MaulingMonkey wrote: post #199

Earlier quoted context omitted.

This is the no true scottsman argument. I mean, no true modern GC. And it's bullshit. Let's be topical and pick on Go since that's the language in the title: https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i... 30% of CPU spent on GC, individual GC pauses already in the milliseconds, despite a tiny half-gig heap in 2019. For gamedev, a single millisecond in the wrong place can be enough to miss vsync and h…

From your link (which I remember reading at the time): > So by simply reducing GC frequency, we saw close to a ~99% drop in mark assist work, which translated to a~45% improvement in 99th percentile API latency at peak traffic. Did you look at the actual article? (Because it doesn't support your point). They added a 10GB memory ballast to keep the GC pacer from collecting too much. That is just a bad heuristic in the…

> Please, we can keep the temperature on the conversation down a bit by just keeping to facts and leaving out a few of these words.

Sure. Let's avoid some of these words too:

> unsupported, tribalist, firmly-held belief that is unsupported by hard data.

Asking for examples is fine and great, but painting broad strokes of the dissenting camp before they have a chance to respond does nothing to help keep things cool.

Re: Generics can make your Go code slower

#234

Post by gmfawcett » Wed, Mar 30, 2022, 9:35 PM UTC

SomeCallMeTim wrote: post #186

gmfawcett wrote: post #63

Earlier quoted context omitted.

That's only 99% of the story. :) Having too many specializations of a C++ template can lead to code bloat, which can degrade cache locality, which can degrade performance.

In my experience, code bloat from templates is overblown. Inlining happens with or without template classes.

That's fair. I guess if you need the functionality in your program, you need the functionality: the codegen approach doesn't matter that much. And like pjmlp said, LTO can make a difference too. Thanks for your thoughts, these kinds of exchanges make me smarter. :)

Re: Generics can make your Go code slower

#235

Post by rowanG077 » Wed, Mar 30, 2022, 9:37 PM UTC

masklinn wrote: post #189

rowanG077 wrote: post #60

Earlier quoted context omitted.

Why is a speed part of the Go language contract but footprint of the executable is not? I, for one, would be quite miffed if an update of the Go compiler would mean an application would no longer fit on my mcu. That is worse then the application running slower.

> Why is a speed part of the Go language contract but footprint of the executable is not? Because footprint of the executable has pretty literally never been, Go has always had deficient DCE and generated huge executables.

It also generates pretty slow executables. That doesn't invalidate the point.

Re: Generics can make your Go code slower

#236

Post by akvadrako » Wed, Mar 30, 2022, 9:39 PM UTC

RedShift1 wrote: post #173

zachruss92 wrote: post #145

For me Go has replaced Node as my preferred backend language. The reason is because of the power of static binaries, the confidence that the code I write today can still run ten years from now, and the performance. The difference in the code I’m working with is being able to handle 250 req/s in node versus 50,000 req/s in Go without me doing any performance optimizations. From my understanding Go was written with dev…

250 vs. 50000 req/s seems like a too big of a difference to me. Sure Go is faster than Node but Node is no slough either, you might want to dig in some deeper why you only got 250 req/s with Node.

That could mostly be due to multithreading. That comes free with go but requires a different model in node.

Re: Generics can make your Go code slower

#237

Post by sophacles » Wed, Mar 30, 2022, 9:40 PM UTC

bborud wrote: post #197

nvarsj wrote: post #46

I'd argue that golang is inherently not a systems language, with its mandatory GC managed memory. I think it's a poor choice for anything performance or memory sensitive, especially a database. I know people would disagree (hence all the DBs written in golang these days, and Java before it), but I think C/C++/Rust/D are all superior for that kind of application. All of which is to say, I don't think it matters. Use t…

Let’s start at the beginning. What is a «systems language» and for what is it typically used?

A systems language is a language used (typically) to write systems :P

Jokes aside, this is kind of a fundamental problem with the term, and many terms around classifying programs. Also worth noting - "program" is a term that is a lot looser than people who typically live above the kernel tend to think.

Re: Generics can make your Go code slower

#238

Post by TheDong » Wed, Mar 30, 2022, 9:45 PM UTC

svnpenn wrote: post #223

TheDong wrote: post #206

Earlier quoted context omitted.

> have you ever actually done that? I have, its not easy. Yes. I do it frequently. "#[derive(Error, Debug)]": https://github.com/dtolnay/thiserror#example Much easier than implementing the error interface in go. Rust is powerful enough to allow macros to remove annoying boiler-plate, and so most people using rust will grab one of the error-handling crates that are de-facto standard and remove the minor pain you're ta…

> Much easier than implementing the error interface in go. is this a joke? You have to import a third party package, just to implement an error interface? Here is Go example, no imports: type errorString string func (e errorString) Error() string { return string(e) }

It was not a joke.

Let's look at a common example: you want to return two different types of errors and have the caller distinguish between them. Let me show it to you in rust and go.

Rust:

    #[derive(Error, Debug)]
    pub enum MyErrors {
       #[error("NotFound: {0}")
       NotFound(String),
       #[error("Internal error")]
       Internal(#[source] anyhow::Error),
    }

The equivalent go would be something like:

    type NotFoundErr struct {
        msg string
    }

    func (err NotFoundErr) Error() string {
        return "NotFound: " + err.msg
    }

    func (err NotFoundErr) Is(target error) bool {
        if target == nil {
            return false
        }
        // All NotFoundErrs are considered the same, regardless of msg
        _, ok := target.(NotFoundErr)
        return ok
    }

    type InternalErr struct {
        wrapped error
    }
    
    func (err InternalErr) Error() string {
        return fmt.Sprintf("Internal error: %s", err.wrapped)
    }

    func (err InternalErr) Unwrap() error {
        return err.wrapped
    }

Re: Generics can make your Go code slower

#239

Post by ninkendo » Wed, Mar 30, 2022, 9:45 PM UTC

titzer wrote: post #114

Karrot_Kream wrote: post #108

Earlier quoted context omitted.

Right. In my experience just taking a few steps (like pre-allocating buffers or arrays) decrease GC pressure enough where GC runs don't actually affect performance enough to matter (as long as you're looking at ~0.5-1 ms P99 response times). But there's always the strident group who says GCs are bad and never offer any circumstance where that could be true.

Indeed. What really kills is extremely high allocation rates and extremely high garbage production. I've seen internal numbers from $megacorp that show that trashy C++ programs (high allocation + deallocation rates) look pretty much the same to CPUs as trashy Java programs, but are far worse in terms of memory fragmentation. Trashy C++ programs can end up spending 20+% of their execution time in malloc/free. That's a…

> I will admit that the programming culture is different for many GC'd languages' communities, sometimes encouraging a very trashy programming style, which contributes to the perception that GC itself is the problem

For some languages (I’m looking at you, Java), there’s not much of a way to program that doesn't generate a bunch of garbage, because only primitives are treated as value types, and for Objects, heap allocations can only be avoided if escape analysis can prove the object doesn’t outlast its stack frame (which isn’t reliable in practice.) (Edit: or maybe it doesn’t happen at all. Apparently escape analysis isn’t used to put objects on the stack even if they are known to not escape: https://www.beyondjava.net/escape-analysis-java)

I honestly can’t imagine much of a way to program in Java that doesn’t result in tremendous GC pressure. You could technically allocate big static buffers and use a completely different paradigm where every function is static and takes data at a defined offsets to said buffers, but… nobody really does this and the result wouldn’t look anything like Java.

Sometimes it’s appropriate to blame the language.

Re: Generics can make your Go code slower

#240

Post by jeffffff » Wed, Mar 30, 2022, 9:45 PM UTC

nvarsj wrote: post #46

I'd argue that golang is inherently not a systems language, with its mandatory GC managed memory. I think it's a poor choice for anything performance or memory sensitive, especially a database. I know people would disagree (hence all the DBs written in golang these days, and Java before it), but I think C/C++/Rust/D are all superior for that kind of application. All of which is to say, I don't think it matters. Use t…

i agree with you that garbage collected languages are bad for systems programming but it's not because garbage collection is inherently bad, it's because gc doesn't handle freeing resources other than memory. for better or worse i've spent most of my professional career writing databases in java and i will not start a database product in java or any other garbage collected language again. getting the error handling and resource cleanup right is way harder in java or go than c++ or rust because raii is the only sane way to do it.