Live data from Hacker News

Speedup from switch to +=

github.com

31–40 of 85 posts

Re: Speedup from switch to +=

#31

If they're seeing these kinds of gains from relatively minor changes to their Python code, I can't help but wonder how much faster the model would run in a compiled language or a language with a good JIT (way more optimization work's gone into the mainstream Javascript runtimes than CPython). I'd assumed that overall performance in Stable Diffusion was limited by the code running on the GPU, with Python performance b…

It can run much faster. For example, using the PyTorch nvFuser JIT gives a 50% speedup:

https://old.reddit.com/r/MachineLearning/comments/xa75km/p_p...

Re: Speedup from switch to +=

#32
post #5

Is python in the fast path? Why not rewrite in a performant language for a XXX% speedup?

The += operator is almost certainly calling some method on sends out the real work to some tuned hardware-specific framework written in a fast language.

Not exactly: most of these frameworks essentially JIT compile the entire operation graph so that it can be executed, and the Python code only touches the data at the endpoints of the full computation. I don't know why the JIT compiler doesn't optimize a = b + a to a += b, but I guess they assumed that the JIT-ed code path would only be used once, so the compiler has to be fast.

Re: Speedup from switch to +=

#33
Whenever I see things like this in highly visible code that people exclaim about across the internet it makes me really take a moment to absorb how much time I spend agonizing over minutae in my daily work and how people who really are just lucky can get away with much worse. Just a reminder about how the idea that "tech" is a meritocracy was never really true.

Re: Speedup from switch to +=

#34

Because of operator overloading "+=" can call a more optimized method than "+". If this code was written in a language without operator overloading I don't think this would be a very interesting pull request. THis could be a example of why some people don't like operator overloading and why some programing languages (java, zig, etc) do not implment the feature.

I don't think this is an operator overloading thing? It's just that `x = y + x` is equivalent to

    z = y + x
    x = z
Basically, creating an object `z` just to throw it away.

`x += y` just adds y to x directly without any intermediary.

You could write this in any language pretty easily. For example, in Rust:

    let x = "abc".to_string();
    let y = "123".to_string();
    let x = x + &y;
as opposed to the more efficient:

    let mut x = "abc".to_string();
    let y = "123".to_string();
    x.push_str(&y);
It's just using an operation to mutate in place vs an immutable operation.

Re: Speedup from switch to +=

#35

Whenever I see things like this in highly visible code that people exclaim about across the internet it makes me really take a moment to absorb how much time I spend agonizing over minutae in my daily work and how people who really are just lucky can get away with much worse. Just a reminder about how the idea that "tech" is a meritocracy was never really true.

I assume that you don't have thousands of people looking over your code, how can you know that it doesn't have similar or greater room for optimization?

Re: Speedup from switch to +=

#36

Because of operator overloading "+=" can call a more optimized method than "+". If this code was written in a language without operator overloading I don't think this would be a very interesting pull request. THis could be a example of why some people don't like operator overloading and why some programing languages (java, zig, etc) do not implment the feature.

If python did not have operator overloading it would not be used for numeric programming to the extent it is. Overloading is key to its success in that field.

The problem is thinking `+' and `+=' are the same, they are not and `+' should not be used when `+=' can be used.

Re: Speedup from switch to +=

#37

If they're seeing these kinds of gains from relatively minor changes to their Python code, I can't help but wonder how much faster the model would run in a compiled language or a language with a good JIT (way more optimization work's gone into the mainstream Javascript runtimes than CPython). I'd assumed that overall performance in Stable Diffusion was limited by the code running on the GPU, with Python performance b…

In PyTorch `x = y + x` is actually semantically different from `x += y`, so you can't easily make the switch with a compiler.

The difference is that `x += y` modifies `x` inplace, where `x = x + y` creates a new object. In other words, if anybody had a reference to `x` before the update, the "optimized" code would break things.

Re: Speedup from switch to +=

#38
This isn't a Python issue, this is a "I'm copying when I don't need to" issue. As I mention elsewhere, you can write this sort of "bug" in almost any language pretty easily (as I demonstrate with Rust).

This isn't a case of "The Python interpreter is bad" it's just that the code is doing what the user asked it to do - create a completely new copy of the data, then overwrite the old copy with it. Immutable operations like this are slow, mutating the value (what += does) is fast.

Granted, a compiled language could recognize that you're doing this, but it also might not - is `+` and `+=` semantically identical such that the compiler can replace one with the other? Maybe? Probably not, if I had to guess. The correct answer is to just use the faster operation, as it is with all language.

I don't know the type of `x`, but I'd suggest another optimization here would be to:

a) Preallocate the buffer rather before mutating it 3x (which is still likely forcing some allocations)

b) Reuse that buffer if it's so important, store it in `self` and clear it before use.

Re: Speedup from switch to +=

#40

If they're seeing these kinds of gains from relatively minor changes to their Python code, I can't help but wonder how much faster the model would run in a compiled language or a language with a good JIT (way more optimization work's gone into the mainstream Javascript runtimes than CPython). I'd assumed that overall performance in Stable Diffusion was limited by the code running on the GPU, with Python performance b…

> way more optimization work's gone into the mainstream Javascript runtimes than CPython

Even so, there are absolutely silly things which can hint JS JITs to optimize (or to not deoptimize). Like defining and instantiating a class rather than just creating POJOs with the same values, or assigning NaN instead of null to uninitialized numeric variables/properties. Conditional control flow can deopt, but generally performs better around different function calls than within a single function. Even creating and throwing errors for control flow (which is generally expensive, and terrible for maintenance) can be optimal if your try/catch is the whole body of the function it resides in. And all of those might vary between JITs.

Post reply on HN