Efficient String Concatenation in Python
skymind.com
Efficient String Concatenation in Python
1–10 of 29 posts
Re: Efficient String Concatenation in Python
#2How would a method 7 using generator expressions fare on the same system, like:
return ''.join(`num` for num in xrange(loop_count))
Re: Efficient String Concatenation in Python
#3Note these tests were run on Python 2.2 which makes the results old enough to almost certainly be invalid. Last time I checked, straight string concatenation was faster than everything else for the types of inputs I was dealing with.
Re: Efficient String Concatenation in Python
#4A very old article from 2004 (please edit the title). The tests are only for cpython2 and for a very old one.
Re: Efficient String Concatenation in Python
#5How would a method 7 using generator expressions fare on the same system, like: return ''.join(`num` for num in xrange(loop_count)) On one hand, it avoids creating a temporary list in memory. On the other, it can't know in advance how long the final output of the loop will be and so couldn't use tricks like preallocating enough RAM.
Interestingly enough, this is pretty much the example given for the timemit module:
http://docs.python.org/2/library/timeit.html
Results:
$ python -m timeit '"-".join(str(n) for n in range(100))'
10000 loops, best of 3: 40.3 usec per loop
$ python -m timeit '"-".join([str(n) for n in range(100)])'
10000 loops, best of 3: 33.4 usec per loop
$ python -m timeit '"-".join(map(str, range(100)))'
10000 loops, best of 3: 25.2 usec per loopRe: Efficient String Concatenation in Python
#6I believe that the current most efficient way of doing this is
''.join(map(str, range(n)))Re: Efficient String Concatenation in Python
#7How would a method 7 using generator expressions fare on the same system, like: return ''.join(`num` for num in xrange(loop_count)) On one hand, it avoids creating a temporary list in memory. On the other, it can't know in advance how long the final output of the loop will be and so couldn't use tricks like preallocating enough RAM.
Does join scan the entire input to know the exact size and allocate that at the start? Doesn't it do the usual thing for resizing and that is to double the capacity? A generator will save a temporary list, it will however result in a lot of overhead for the generator switches. It might be beneficial if the input is really big.
Re: Efficient String Concatenation in Python
#8Note that since this article was written (2004) CPython performs an in-place optimisation for assignments of the form
s1 += s2
Re: Efficient String Concatenation in Python
#9Note that since this article was written (2004) CPython performs an in-place optimisation for assignments of the form s1 += s2 There are details at point six of http://docs.python.org/2/library/stdtypes.html#sequence-type... , where it also says that str.join() is preferable.
Does PyPy have the same heuristic? If not, I wouldn't recommend relying on it.
Re: Efficient String Concatenation in Python
#10Note that since this article was written (2004) CPython performs an in-place optimisation for assignments of the form s1 += s2 There are details at point six of http://docs.python.org/2/library/stdtypes.html#sequence-type... , where it also says that str.join() is preferable.
Does PyPy have the same heuristic? If not, I wouldn't recommend relying on it.
If not, I would recommend submitting a bug to PyPy.