Live data from Hacker News

Fast Python loops

python.org

11–20 of 54 posts

Re: Fast Python loops

#11
Virtually every discussion of Python and performance should mention PyPy. If for no other reason than to disqualify it "doesn't work with deployment feature x or customer requirement y."

It's often a "free" speedup and generally Just Works.

Re: Fast Python loops

#12
post #3

Interesting read! I'm still missing what "B" in the Python code referring to as well.

The python timings are for 1000 iterations. You are currently comparing apple to 1000 apples :)

Re: Fast Python loops

#13
post #4

I wonder if bytearray([97,98,99]).decode('latin-1') is faster still. Edit: Yup, it is. 3x faster. https://gist.github.com/anonymous/18e372e8d0173e77b5c405920d...

Was thinking exactly the same. Plus you save an import. And no need to wrap it in a function since it's a short method call on a built-in. Would bytearray([97,98,99]).decode('ascii') be even faster ?

Re: Fast Python loops

#14
post #4

I wonder if bytearray([97,98,99]).decode('latin-1') is faster still. Edit: Yup, it is. 3x faster. https://gist.github.com/anonymous/18e372e8d0173e77b5c405920d...

Was thinking exactly the same. Plus you save an import. And no need to wrap it in a function since it's a short method call on a built-in. Would bytearray([97,98,99]).decode('ascii') be even faster ?

It would fail when you pass it 128. He's passing in values from 0-255. "Ascii" is sort of a misnomer in his post.

Re: Fast Python loops

#15
post #9

No list comprehension? I thought list comprehension were faster than loops?

Yes but it's still a loop in Python, which means it calls iter() then __next__() in a try/except to catch StopIteration in the end. It's a lot of overhead.

tostring() or encode() will do an implicity loop in C, which will be much faster.

Re: Fast Python loops

#16
post #6

The author states: "There's a general technique to avoid quadratic behavior in algorithms like this. I coded it as follows for strings of exactly 256 items:" def f5(list): string = "" for i in range(0, 256, 16): # 0, 16, 32, 48, 64, ... s = "" for character in map(chr, list[i:i+16]): s = s + character string = string + s return string I am not understanding what the technique is or why using a step size of 16 in the…

A wild guess on what he is trying to do:

- Use Vector Processor instead of Scalar Processor, SIMD(SSE/AVX/etc) has 16 128-bit registers. In the wikipedia entry of SSE it says that SSE2 has support for 16 chars.

Things that don't add up:

- This should be a compiler optimization

- This makes something readable unnecessarily complex

- s is mutating

- You could eliminate the inner loop and s

Re: Fast Python loops

#18
post #12
post #3

Interesting read! I'm still missing what "B" in the Python code referring to as well.

The python timings are for 1000 iterations. You are currently comparing apple to 1000 apples :)

You mean 100 iterations instead of 1000?

I have noted "took 0.000183 (For one iteration)" :)

Re: Fast Python loops

#19
post #7

According to archive.org this essay is from at most June 2006, which would put it at Python 2.4 or earlier. The specific performance characteristics of Python will have changed greatly in the intervening period.

Thanks for finding this out, it explains a lot. IIRC, the str += optimization was added in 2.6 which would have made f1 quite fast.

Re: Fast Python loops

#20
A big problem here, though, is that

> array.array('B', list).tostring()

is not terribly readable or beautiful.

In fact, I think I'd need a comment to explain that it's typecasting elements of a list strings to ints.

Optimizations like this are an identity crisis for Python (and have been for a long time, as evinced by the age of this essay): Is it focused on being human-readable and otherwise compliant with the Zen of Python?

Or is it focused on exposing high-performance variants for every way of doing something?

...and while it may be tempting to say that the answer is "both," this removes "one obvious way" as a lingual basis for Python.

Ultimately, there needs to be a reconciliation wherein these high-performance variants are folded back in (even if it means cheating in the implementation) to the idiomatic expressions.

Type hints and the evolving async syntax are additional considerations in this arena.

I know that this has been a topic of conversation at the language summit in the past; I surmise it will be very much so next month.

Post reply on HN