Live data from Hacker News

Incrementally improving the performance of a Python script

mycode.doesnot.run

11–20 of 24 posts

Re: Incrementally improving the performance of a Python script

#11
Python has some unusual performance behaviors. IIRC you can also speed up the performance of your program a lot by assigning intermediate variables instead of referencing properties, for example:

[A.b[i] for i in range(100)]

is a lot slower than:

B = A.b

[B[i] for i in range (100)]

Re: Incrementally improving the performance of a Python script

#12

Found something weird with codes posted in the links. I am getting different outputs for the intermediate and the final codes for the input of "5; 1,2,3,4,4". Can someone help? 1. https://imgur.com/a/u8O65AF 2. https://imgur.com/a/uHniZof

Got it. The input is not valid for the problem.

Re: Incrementally improving the performance of a Python script

#13

Found something weird with codes posted in the links. I am getting different outputs for the intermediate and the final codes for the input of "5; 1,2,3,4,4". Can someone help? 1. https://imgur.com/a/u8O65AF 2. https://imgur.com/a/uHniZof

> Starting from an array A that has n distinct integers I don't know in what ways they are differents, but these programs were not designed to work with duplicates in the input. This probably explains the results.

Yeah you're right

Re: Incrementally improving the performance of a Python script

#14

Python has some unusual performance behaviors. IIRC you can also speed up the performance of your program a lot by assigning intermediate variables instead of referencing properties, for example: [A.b[i] for i in range(100)] is a lot slower than: B = A.b [B[i] for i in range (100)]

Unusual is a strange qualifier here. Removing a dereference leading to a speedup is probably one of the few almost universal optimizations.

Re: Incrementally improving the performance of a Python script

#16

Python has some unusual performance behaviors. IIRC you can also speed up the performance of your program a lot by assigning intermediate variables instead of referencing properties, for example: [A.b[i] for i in range(100)] is a lot slower than: B = A.b [B[i] for i in range (100)]

I'm not sure about other implementations, but "optimizing" CPython ends up being optimizing against counter-intuitive interpreter internals rather than time-complexity of the code.

For example, in CPython 3.6,

  n = 0
  d = 100
  for i in range(10**6):
      n += i
      if n >= d:
          n %= d
is slower than

  n = (n + i) % d
This counter to lower level languages, where dividing by a variable is costly, and the CPU can predict the pipeline to be false most of the time in the conditional and thus skip it.

Re: Incrementally improving the performance of a Python script

#17

Python has some unusual performance behaviors. IIRC you can also speed up the performance of your program a lot by assigning intermediate variables instead of referencing properties, for example: [A.b[i] for i in range(100)] is a lot slower than: B = A.b [B[i] for i in range (100)]

Unusual is a strange qualifier here. Removing a dereference leading to a speedup is probably one of the few almost universal optimizations.

People do it in C# a lot with Count and Length on Lists and Arrays and it ruins array bounds check elision.

It is something that compilers do for you, in this context, in most programming environments.

Re: Incrementally improving the performance of a Python script

#18

Earlier quoted context omitted.

Unusual is a strange qualifier here. Removing a dereference leading to a speedup is probably one of the few almost universal optimizations.

People do it in C# a lot with Count and Length on Lists and Arrays and it ruins array bounds check elision. It is something that compilers do for you, in this context, in most programming environments.

Unfortunately, Python being hopelessly dynamic, `A.b` could involve executing arbitrary code and side effects depending on what `A` is. So hoisting `A_b = A.b` out of the loop may produce different results and thus can't be done automatically without some kind-of incredibly (impossibly?) smart static analysis :(

(though perhaps a JIT could perform this optimization for the usual, non-surprising path)

Re: Incrementally improving the performance of a Python script

#19

Python has some unusual performance behaviors. IIRC you can also speed up the performance of your program a lot by assigning intermediate variables instead of referencing properties, for example: [A.b[i] for i in range(100)] is a lot slower than: B = A.b [B[i] for i in range (100)]

Unusual is a strange qualifier here. Removing a dereference leading to a speedup is probably one of the few almost universal optimizations.

It isn't the removed dereference that speeds it up though, it is placing the pointer into the local scope, which is searched before the global scope.

Re: Incrementally improving the performance of a Python script

#20

Python has some unusual performance behaviors. IIRC you can also speed up the performance of your program a lot by assigning intermediate variables instead of referencing properties, for example: [A.b[i] for i in range(100)] is a lot slower than: B = A.b [B[i] for i in range (100)]

[deleted]
Post reply on HN