Speedup from switch to +=

I'm wondering, because recent version have improved performance a lot. 3.11 is much faster than 3.10, and what's in 3.12 is already much faster than 3.11.

Re: Speedup from switch to +=

#5

Post by dahfizz » Mon, Sep 12, 2022, 3:13 AM UTC

Is python in the fast path? Why not rewrite in a performant language for a XXX% speedup?

Re: Speedup from switch to +=

#6

Post by onedognight » Mon, Sep 12, 2022, 3:13 AM UTC

mhzsh wrote: post #3

But why is it faster? A non-associative translation to byte code (or however python works)?

My guess is that it operates in place with no memory allocations or copying.

Re: Speedup from switch to +=

#7

Post by JonathonW » Mon, Sep 12, 2022, 3:16 AM UTC

If they're seeing these kinds of gains from relatively minor changes to their Python code, I can't help but wonder how much faster the model would run in a compiled language or a language with a good JIT (way more optimization work's gone into the mainstream Javascript runtimes than CPython).

I'd assumed that overall performance in Stable Diffusion was limited by the code running on the GPU, with Python performance being a fairly minor factor-- but I guess that's not the case?

Re: Speedup from switch to +=

#8

Post by actually_a_dog » Mon, Sep 12, 2022, 3:16 AM UTC

mhzsh wrote: post #3

But why is it faster? A non-associative translation to byte code (or however python works)?

Not exactly:

    >>> def f(x): x += 1
    ... 
    >>> def g(x): x = x + 1
    ... 
    >>> dis.dis(f)
      1           0 LOAD_FAST                0 (x)
                  3 LOAD_CONST               1 (1)
                  6 INPLACE_ADD         
                  7 STORE_FAST               0 (x)
                 10 LOAD_CONST               0 (None)
                 13 RETURN_VALUE        
    >>> dis.dis(g)
      1           0 LOAD_FAST                0 (x)
                  3 LOAD_CONST               1 (1)
                  6 BINARY_ADD          
                  7 STORE_FAST               0 (x)
                 10 LOAD_CONST               0 (None)
                 13 RETURN_VALUE

Re: Speedup from switch to +=

#9

Post by lnyan » Mon, Sep 12, 2022, 3:17 AM UTC

mhzsh wrote: post #3

But why is it faster? A non-associative translation to byte code (or however python works)?

For PyTorch, `+=` is interpreted as an in-place operation

Re: Speedup from switch to +=

#10

Post by eminence32 » Mon, Sep 12, 2022, 3:18 AM UTC

eru wrote: post #4

I wonder what version of Python they were using? I'm wondering, because recent version have improved performance a lot. 3.11 is much faster than 3.10, and what's in 3.12 is already much faster than 3.11.

The upstream Stable Diffusion uses python 3.8:

https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a...