Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python extensions should be lazy

gauge.sh

Post reply on HN
Print view View original

31–40 of 65 posts

Re: Python extensions should be lazy

#31

Post by dumah »

f33d5173 wrote: post #24
tomjakubowski wrote: post #6

Earlier quoted context omitted.

re: 3, Python has a native numeric array type https://docs.python.org/3/library/array.html

array is for serializing to/from binary data. It isn't useful for returning from a library because the only way a python programmer can consume it is by converting into python objects, at which point there is no efficiency benefit. numpy has a library of functions for operating directly on the referenced data, as well as a cottage industry of libraries that will take a numpy array as input. Obviously someone might en…

Cython implements a C api for accessing the underlying data structure.

Arrays implement the buffer interface so they can be used efficiently with tools like numpy.

Re: Python extensions should be lazy

#32

Post by pdhborges »

How much time did the PyAST_mod2obj actually take? The rewritte is 16x faster but the article doesn't make it clear if most of the speedup came from switching to the ruff parser (specially because it puts the GC overhead at only 35% of the runtime).

Re: Python extensions should be lazy

#33

Post by Spivak »

RhysU wrote: post #22
lalaland1125 wrote: post #2

Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. The key for optimizing a Python extension is to minimize the number of times you have to interact with Python. A couple of other tips in addition to what this article provides: 1. Object pooling is quite useful as it can significantly cut down on the number of allocations. 2. Be very care…

> Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. I have always loved how the trick to making Python better eventually comes down to not writing Python.

Is this not expected? You're never going to have any language with the kind of dynamism that Python/Ruby/JS have while also having performant number crunching simply because Python has to do more significantly more work for the same line of code. You could envision a world where a JIT could recognize cases where all that dynamism falls away and you can generate code similar to what you would get in the equivalent C but that's just a fancy way to not write Python again. You would be writing in this informal not super well defined restricted subset of Python that JITs cleanly.

The problem time immortal is language complexity vs the ability to hint to your compiler that it can make much stronger assumptions about your code than it has to assume naturally which is where we got __slots__. And there's lots of easy wins you could get in Python that eliminate a significant amount of dynamism-- you could tell your compiler that you'll never shadow names for example, that this list is fixed size, that you don't want number promotion but they all require adding stuff to the language to indicate that.

When you're looking from the bottom up you end up making different trade-offs. Because while you get nice primitives that generate very tight assembly when you need that dynamism you end up having this object model that exists in the abstract that you orchestrate and poke at but don't really see, like gobject. Ironically, HN's love-to-hate language C++ gives you both simultaneously but at the cost of a very complicated language.

Re: Python extensions should be lazy

#35

Post by garbagepatch »

jay-barronville wrote: post #19

Evan, just a tip… When linking to code on GitHub in an article like this, for posterity, it’s a good idea to link based on a specific commit instead of a branch. It might be a good idea to change your link to the `Py_CompileStringObject()` function in CPython’s `Python/pythonrun.c` [0] to a commit-based link [1]. [0]: https://github.com/python/cpython/blob/main/Python/pythonrun... [1]: https://github.com/python/cpyth…

Tip: in Github, press `y` when looking at a `/blob/branch/something` url to turn it into the current commit.

Re: Python extensions should be lazy

#36

Post by 0x63_Problems »

pdhborges wrote: post #32

How much time did the PyAST_mod2obj actually take? The rewritte is 16x faster but the article doesn't make it clear if most of the speedup came from switching to the ruff parser (specially because it puts the GC overhead at only 35% of the runtime).

That's a good question. I don't have an easy way to rerun the comparison since this happened actually a while ago, but I do remember some relevant numbers.

In the first iteration of the Rust extension, I actually used the parser from RustPython. Although I can't find it at the moment, I think the RustPython parser was actually benchmarked as worse than the builtin ast parse (when both returned Python objects).

Even with this parser, IIRC the relevant code was around 8-11x faster when it avoided the Python objects. Apart from just the 35% spent in GC itself, the memory pressure appeared to be causing CPU cache thrashing (`perf` showed much poorer cache hit rates). I'll admit though that I am far from a Valgrind expert, and there may have been another consequence of the allocations that I missed!

Re: Python extensions should be lazy

#37

Post by lifthrasiir »

lalaland1125 wrote: post #2

Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. The key for optimizing a Python extension is to minimize the number of times you have to interact with Python. A couple of other tips in addition to what this article provides: 1. Object pooling is quite useful as it can significantly cut down on the number of allocations. 2. Be very care…

> 2. Be very careful about tools like pybind11 that make it easier to write extensions for Python. They come with a significant amount of overhead. For critical hotspots, always use the raw Python C extension API.

Agrees on the broader point (and I don't like pybind11 that much anyway), but the raw Python C extension API is often hard to use correctly. I would suggest that you should at least have a rough idea about how higher-level libraries like pybind11 would translate to the C API, so that you can recognize performance pitfalls in advance.

> 3. Use numpy arrays whenever possible when returning large lists to Python. A python list of python integers is amazingly inefficient compared to a numpy array of integers.

Or use the `array` module in the standard library if that matters. numpy is not a small library and has quite a large impact on the initial startup time. (Libraries like PyTorch are even much worse to be fair, though.)

Re: Python extensions should be lazy

#38

Post by [deleted] »

stabbles wrote: post #5
lalaland1125 wrote: post #2

Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. The key for optimizing a Python extension is to minimize the number of times you have to interact with Python. A couple of other tips in addition to what this article provides: 1. Object pooling is quite useful as it can significantly cut down on the number of allocations. 2. Be very care…

Why not go all the way and limit the times you have to interact with Python to zero ;)

[deleted]

Re: Python extensions should be lazy

#39

Post by lifthrasiir »

hansvm wrote: post #4
0x63_Problems wrote: post #3

Earlier quoted context omitted.

Totally agree, keeping the interface with the extension as thin as possible makes sense. I hadn't considered object pooling in this context, it might be more involved since each node has distinct data but for my use case it might still be a performance win. Have you ever used pyo3 for rust bindings? I haven't measured the overhead but I have been assuming that it's worth the tradeoff vs. rolling my own. (I'm the auth…

My last workplace used pyo3 for a project. It was slower than vanilla Python, and you picked up all the normal compiled-language problems like slow builds and cross-compilation toolchains. I wouldn't take away from that observation that pyo3 is slow (it was just a poor fit; FFI for miniscule amounts of work), but the fact that the binding costs were higher than vanilla Python computations suggests that the overhead i…

That's pretty surprising, because I have also extensively used PyO3 in my daily job and it was quite performant. Your comment does seem to suggest that you were also using the `numpy` crate or similar in addition to `pyo3`, which performance might be more variable than I would expect for PyO3 though. (I personally minimized the use of `numpy` for that reason, but didn't have a particular performance issue with it anyway.)

Re: Python extensions should be lazy

#40

Post by im3w1l »

ameliaquining wrote: post #28
coldtea wrote: post #17

Earlier quoted context omitted.

Even better if Python adds a mainline pandas/numpy like C-based table structure, with a very small subset of the pandas/numpy functionality, that's also convertable to pandas/numpy/etc.

What kind of subset would you have in mind? I think that any kind of numeric operation would be off the table, for the reasons given in PEP 465: "Providing a quality implementation of matrix multiplication is highly non-trivial. Naive nested loop implementations are very slow and shipping such an implementation in CPython would just create a trap for users. But the alternative – providing a modern, competitive matrix…

Numpy is incredibly widespread and basically a standard so I would propose: It should have exactly the same layout in memory as a numpy array. It's fine if it has a very limited set of operations out-of-the-box. Maybe something like get, set, elementwise-arithmetic. Work with numpy project to make it possible to cast it to numpy array to help the common case where someone is fine with a dep on numpy and wants the full set of numpy operations.
Post reply on HN