Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How many lines of C it takes to execute a + b in Python

codeconfessions.substack.com

Post reply on HN
Print view View original

31–40 of 224 posts

Re: How many lines of C it takes to execute a + b in Python

#31

Post by masklinn »

tyingq wrote: post #6
andai wrote: post #4

A while back someone posted their patch to cpython where they replaced the hash function with a fast one and claimed this dramatically sped up the whole Python runtime. They claimed that the hash function was used constantly —e.g. 11 times in print("hello world")—because it's used to look up object properties. Apparently the default implementation is not optimized for performance but for security, just in case the so…

It looks like you can disable the slower randomized hashing yourself by setting PYTHONHASHSEED to 0. Though I don't know if there's further speedup to be had by using a different hash implementation. https://docs.python.org/3/using/cmdline.html#envvar-PYTHONHA... The original issue: https://bugs.python.org/issue13703

That only disables the keying of the hash function (sets the initial value to 0), it does not change the hash function.

The hashseed is a per-process value, it has basically no impact on performances.

Re: How many lines of C it takes to execute a + b in Python

#32

Post by mgl »

A more thorough comparison of time, energy and memory required to execute similar Python, C, Rust, Java, C#, etc. code:

https://stratoflow.com/efficient-and-environment-friendly-pr...

And why Mojo could be an answer for high (well, higher) performance Python: https://stratoflow.com/introduction-to-mojo-programming-lang...

Re: How many lines of C it takes to execute a + b in Python

#33

Post by jasode »

Dwedit wrote: post #20

Lines of C aren't the defining factor. You want total instructions executed.

>Lines of C aren't the defining factor. You want total instructions executed.

I understand what your clarification is trying to provide but it isn't relevant to this particular thread's article. The article is not about "performance benchmarks" where you need cpu instructions as a definitive unit-of-measure for comparisons.

Instead of measuring performance, the author's theme in this case is more akin to "decompiling" or "reverse-engineering". He takes a tiny piece of Python code and then maps it back to the actual CPython source *.c and *.h files that implements the Python vm. He added several deep links to the relevant sections of CPython source code on Github to help illustrate the mappings between Python's BINARY_OP to the .c and .h files. The article is sharing the type of knowledge you'd gain by loading up CPython in a debugger and single-stepping through the source code line-by-line.

In other words, the article's title could also have been: "Which Lines of CPython does it Take to Execute a + b in Python?"

For the scope of this particular article, the "lines of C" _are_ the defining factor because the subject of dissection is CPython's .c/.h files.

Re: How many lines of C it takes to execute a + b in Python

#34

Post by Asraelite »

andai wrote: post #4

A while back someone posted their patch to cpython where they replaced the hash function with a fast one and claimed this dramatically sped up the whole Python runtime. They claimed that the hash function was used constantly —e.g. 11 times in print("hello world")—because it's used to look up object properties. Apparently the default implementation is not optimized for performance but for security, just in case the so…

This may be a somewhat uninformed opinion, but I think CPython is just straight up not particularly good software. There are a million and one optimizations that other major scripting runtimes (V8, LuaJIT, PyPy, Ruby YJit etc.) have had for years that CPython is lacking. This is by design though. CPython has never been focused on performance, that's why it's not even JIT. It optimizes for simplicity and easy interoperability with C.

The problem is that because Python is such a ubiquitous language, CPython gets more attention than it deserves. People see it as an archetypical implementation of a scripting language. We get blogposts like this examining its inner workings, discussions about how its performance could be improved, comparisons of its speed vs. compiled languages, and tutorials on how to optimize code to run faster in it. I feel like all of this effort would be better spent on discussions about runtimes that actually try to be fast.

Re: How many lines of C it takes to execute a + b in Python

#35

Post by geertj »

The Python C API has become so verbose that I recommend against using it directly. Recently I mapped a few external libraries using Nanobind, and the result was significantly more concise. Nanobind uses the type system of modern C++ (17 and above) to provide most argument conversions, Mapping C functions is typically just a few lines of code. It also provides a great way to map C++ constructs such as classes, exceptions, and standard containers.

Nanobind is by the same author that started pybind11, used by Tensorflow and PyTorch. The web site [1] contains a bit more of the rationale.

[1] https://nanobind.readthedocs.io/en/latest/why.html

Re: How many lines of C it takes to execute a + b in Python

#36

Post by masklinn »

petters wrote: post #15
andai wrote: post #4

A while back someone posted their patch to cpython where they replaced the hash function with a fast one and claimed this dramatically sped up the whole Python runtime. They claimed that the hash function was used constantly —e.g. 11 times in print("hello world")—because it's used to look up object properties. Apparently the default implementation is not optimized for performance but for security, just in case the so…

Do you mean this post? https://www.reddit.com/r/Python/s/raofvsKCiz That speedup was disputed in the comments. I haven’t tried it myself though

https://www.reddit.com/r/Python/comments/mgi4op/comment/gswg...

I’d use “thoroughly disproven” rather than “disputed”.

I’m sure the hash function can be changed, but as various comments noted:

- the benchmark was nonsensical

- cpython caches string hashes, and “symbols” are interned, so outside of dynamic attribute access from dynamically constructed strings each hash for attribute purposes or namespace lookup is computed once per process

- and finally (though probably not the biggest issue) xxhash is known for being mostly useful on larger sizes (>128 bytes, although you can find better hashes (city IIRC) up to 512 or so)

Much like Rust, CPython uses siphash as its default, it’s a pretty good all rounder though not the fastest. It actually used to use FNV before HashDOS.

CPython does suffer from the inability of users to configure hash functions since it’s an object property rather than a container property.

Re: How many lines of C it takes to execute a + b in Python

#38

Post by 082349872349872 »

fractalb wrote: post #26
unnah wrote: post #17

Earlier quoted context omitted.

Yes, but C is a much more arcane and complicated language. Case in point: instead of writing a+b in Python, in C you would have to write a+b (Jokes aside, it would really be more complicated in C if a and b were actually strings or lists.)

> it would really be more complicated in C if a and b were actually strings or lists You don't event need strings or lists for that. Just imagine bigger numbers for a and b. Arbitrarily long integer addition is not a native language feature in C. edit: formatting

to be concrete:

    fac = lambda n: 1 if n
is 3 lines of python; how many lines of C code would it take to execute?

Re: How many lines of C it takes to execute a + b in Python

#39

Post by tgv »

082349872349872 wrote: post #38
fractalb wrote: post #26

Earlier quoted context omitted.

> it would really be more complicated in C if a and b were actually strings or lists You don't event need strings or lists for that. Just imagine bigger numbers for a and b. Arbitrarily long integer addition is not a native language feature in C. edit: formatting

to be concrete: fac = lambda n: 1 if n is 3 lines of python; how many lines of C code would it take to execute?

One: return 1.7112245243e98

Joking aside: these are not catch-all comparisons. Nor is the article. But Python is mucher slower and much safer than C. It's easier to start in Python than in C.

Re: How many lines of C it takes to execute a + b in Python

#40

Post by [deleted] »

Asraelite wrote: post #34
andai wrote: post #4

A while back someone posted their patch to cpython where they replaced the hash function with a fast one and claimed this dramatically sped up the whole Python runtime. They claimed that the hash function was used constantly —e.g. 11 times in print("hello world")—because it's used to look up object properties. Apparently the default implementation is not optimized for performance but for security, just in case the so…

This may be a somewhat uninformed opinion, but I think CPython is just straight up not particularly good software. There are a million and one optimizations that other major scripting runtimes (V8, LuaJIT, PyPy, Ruby YJit etc.) have had for years that CPython is lacking. This is by design though. CPython has never been focused on performance, that's why it's not even JIT. It optimizes for simplicity and easy interope…

[deleted]
Post reply on HN