Most of the time I find the pros of not mutating variables out weight any potential memory / performance gain, of course it depends on what you're doing, but I find it rare other than perhaps scientific related code.
Performance hacks for faster Python code
61–65 of 65 posts
Re: Performance hacks for faster Python code
#62Funny that Hack #1 compares list versus set value lookup, but the timer doesn't include the time to copy the list into a set. Hack #2 warns against unnecessary copying, and the time for copying the list is almost the same as the performance gain in Hack #1.
In fact, creating a set takes longer than copying a list since it requires hash insertion, so it's actually much faster to do the opposite of what they suggest for #1 (in the case of a single lookup, for this test case).
Here's the results with `big_set = set(big_list)` inside the timing block for the set case:
List lookup: 0.013985s
Set lookup: 0.052468sRe: Performance hacks for faster Python code
#63All good suggestions. However, the last one should have also mentioned functools.cache and functools.lru_cache.
Re: Performance hacks for faster Python code
#64Maybe also knowing when not to use python, or finding a solution in python that uses C/rust/etc underneath.
maybe you can skip C and just use assembly
If it's a module doing cryptography, maybe...Not suggesting C or rust in places where it doesn't make sense. Python itself make choices about whether the things it ships with are pure python or not. They often are not.
Re: Performance hacks for faster Python code
#65Funny that Hack #1 compares list versus set value lookup, but the timer doesn't include the time to copy the list into a set. Hack #2 warns against unnecessary copying, and the time for copying the list is almost the same as the performance gain in Hack #1. In fact, creating a set takes longer than copying a list since it requires hash insertion, so it's actually much faster to do the opposite of what they suggest fo…
I'm surprised that this was not the first complaint. It's pretty clear "Hack" #1 isn't going to work.
import random
import time
def timeit(func, _list, n=1000):
start = time.time()
for _ in range(n):
func(_list=_list )
end = time.time()
print(f"Took {end-start} s")
return
def lstsearch(_list):
sf = random.randint(0,len(_list))
if sf in _list:
return
return
def setsearch(_list):
sf = random.randint(0, len(_list))
if sf in set(_list):
return
return
mylist = list(range(100000))
timeit(lstsearch, mylist)
timeit(setsearch, mylist)
----
Took 0.23349690437316895 s
Took 0.8901607990264893 s