Why not just:
def f(list):
return ''.join([chr(l) for l in list])Python Patterns - An Optimization Anecdote
21–30 of 45 posts
Re: Python Patterns - An Optimization Anecdote
#22Why not just: def f(list): return ''.join([chr(l) for l in list])
This would be my default approach as well. I'd be interested to see how it compares-- my guess is that it would be equivalent to or slightly faster than f6(), and slower than the version that uses the array library.
Re: Python Patterns - An Optimization Anecdote
#23I hope that one day soon our compilers will be smart enough to optimize simple things like this automatically, so we can focus more on writing for clarity. This seems like exactly the kind of thing a next-gen compiler should be able to optimize.
I'm not sure it's so possible, considering the dynamic nature of Python. Consider one of the biggest problems - dynamic lookups. Calling a function like "chr", which is looked up dynamically and found in the global scope each time, takes a lot of time. Just adding a local variable which has the same value (i.e. lchr = chr) already gives a speedup. How can an optimizer fix this? Technically, a new lookup must be perfo…
> Calling a function like "chr", which is looked up dynamically
Shouldn't that be just one or two pointer-dereferences? Or is Python putting globals in hash-table and looking them up each and every time?
Re: Python Patterns - An Optimization Anecdote
#24Why not just: def f(list): return ''.join([chr(l) for l in list])
Because it's an old article. Admittedly, there's no easy way to know this, because it has no date, but believe me, it's been around for ages. At the very least, it predates the silly str.join method (and new-style classes and string methods altogether).
However, before that, there was the string module, which had the join() function. This article probably relates to it in some way; I think that either it predates string.join (I'm not sure when that was added to Python, but it's been around since at least 1.4), or it clarifies how it was implemented.
The author is Guido van Rossum, by the way (another thing that isn't mentioned in the article).
Re: Python Patterns - An Optimization Anecdote
#25Earlier quoted context omitted.
I'm not sure it's so possible, considering the dynamic nature of Python. Consider one of the biggest problems - dynamic lookups. Calling a function like "chr", which is looked up dynamically and found in the global scope each time, takes a lot of time. Just adding a local variable which has the same value (i.e. lchr = chr) already gives a speedup. How can an optimizer fix this? Technically, a new lookup must be perfo…
> Calling a function like "chr", which is looked up dynamically Shouldn't that be just one or two pointer-dereferences? Or is Python putting globals in hash-table and looking them up each and every time?
The latter; otherwise it wouldn't be dynamic. :-)
I don't know the details, but I assume that some of the Python compiler projects do make assumptions like this, i.e. they can compile to faster code if you don't shadow or outright overwrite builtins, etc.
Re: Python Patterns - An Optimization Anecdote
#26Earlier quoted context omitted.
> Calling a function like "chr", which is looked up dynamically Shouldn't that be just one or two pointer-dereferences? Or is Python putting globals in hash-table and looking them up each and every time?
The latter; otherwise it wouldn't be dynamic. :-) I don't know the details, but I assume that some of the Python compiler projects do make assumptions like this, i.e. they can compile to faster code if you don't shadow or outright overwrite builtins, etc.
I still don't see why the variable names can't be interned somehow by the parser. This would mean that a global variable would be just a pointer in to the table, and "lookup" would be just dereferencing that pointer. AFAICT, this is how symbols and packages work in Common Lisp.
Re: Python Patterns - An Optimization Anecdote
#27Why not just: def f(list): return ''.join([chr(l) for l in list])
It is actually slower. Adding an f8 function using the ''.join() method:
sikanda-13:30:02:/tmp$ python f.py
[...]
f1 0.07
f2 0.12
f3 0.07
f4 0.06
f5 0.09
f6 0.04
f7 0.02
f8 0.07
Re: Python Patterns - An Optimization Anecdote
#28Why not just: def f(list): return ''.join([chr(l) for l in list])
This would be my default approach as well. I'd be interested to see how it compares-- my guess is that it would be equivalent to or slightly faster than f6(), and slower than the version that uses the array library.
From the article, I'm guessing the author was using a version Incidentally, I ran my function and f6 from the article on a list of 9600 integers using Python 2.6.3. f6 ran in 0.00303250832159 seconds and my function ran in 0.00421450212248 seconds.
Re: Python Patterns - An Optimization Anecdote
#29I once found myself writing a python program to calculate Poker hands. It was pretty fun to write - it generated, for any given hand you hold, every possible way the game can go forward, and generated statistics about how many times you win (against 1 opponent only, who could be holding anything.) The program initially ran for 20 minutes on each hand. I worked a day on optimizing it, and got it down to around 1-2 min…
Point 3 is weakened by the fact that map can indeed be faster. http://codepad.org/SzXcubXy -- in fact, point 2 and point 3 are in conflict, because map() does use a C-loop, whereas list comprehensions do not. As it turns out, point 2 was the correct one.
Also, and this is just on a more minor, pedantic note, list comprehensions basically are regular for loops, except with the use of the list-append opcode instead of a getattr-call pair. This produces a minor speedup, but it's nothing to write home about, honest. (using a different pastebin because it has a feature I needed): http://ideone.com/dT1LG . The story is a bit different on 3.x, where list comps are implemented very differently: http://ideone.com/58FGZ
Re: Python Patterns - An Optimization Anecdote
#30Why not just: def f(list): return ''.join([chr(l) for l in list])
This would be my default approach as well. I'd be interested to see how it compares-- my guess is that it would be equivalent to or slightly faster than f6(), and slower than the version that uses the array library.
Actually, map is still faster:
In [24]: timeit ''.join(map(chr, ll))
100000 loops, best of 3: 3.91 us per loop
In [25]: timeit ''.join([chr(i) for i in ll])
100000 loops, best of 3: 5.85 us per loop
In [26]: timeit ''.join(chr(i) for i in ll)
100000 loops, best of 3: 7.79 us per loop