Python extensions should be lazy
gauge.sh
Python extensions should be lazy
1–10 of 65 posts
Re: Python extensions should be lazy
#2Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments.
The key for optimizing a Python extension is to minimize the number of times you have to interact with Python.
A couple of other tips in addition to what this article provides:
1. Object pooling is quite useful as it can significantly cut down on the number of allocations.
2. Be very careful about tools like pybind11 that make it easier to write extensions for Python. They come with a significant amount of overhead. For critical hotspots, always use the raw Python C extension API.
3. Use numpy arrays whenever possible when returning large lists to Python. A python list of python integers is amazingly inefficient compared to a numpy array of integers.
Re: Python extensions should be lazy
#3Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. The key for optimizing a Python extension is to minimize the number of times you have to interact with Python. A couple of other tips in addition to what this article provides: 1. Object pooling is quite useful as it can significantly cut down on the number of allocations. 2. Be very care…
Totally agree, keeping the interface with the extension as thin as possible makes sense.
I hadn't considered object pooling in this context, it might be more involved since each node has distinct data but for my use case it might still be a performance win.
Have you ever used pyo3 for rust bindings? I haven't measured the overhead but I have been assuming that it's worth the tradeoff vs. rolling my own.
(I'm the author)
Re: Python extensions should be lazy
#4Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. The key for optimizing a Python extension is to minimize the number of times you have to interact with Python. A couple of other tips in addition to what this article provides: 1. Object pooling is quite useful as it can significantly cut down on the number of allocations. 2. Be very care…
Totally agree, keeping the interface with the extension as thin as possible makes sense. I hadn't considered object pooling in this context, it might be more involved since each node has distinct data but for my use case it might still be a performance win. Have you ever used pyo3 for rust bindings? I haven't measured the overhead but I have been assuming that it's worth the tradeoff vs. rolling my own. (I'm the auth…
My last workplace used pyo3 for a project. It was slower than vanilla Python, and you picked up all the normal compiled-language problems like slow builds and cross-compilation toolchains.
I wouldn't take away from that observation that pyo3 is slow (it was just a poor fit; FFI for miniscule amounts of work), but the fact that the binding costs were higher than vanilla Python computations suggests that the overhead is (was?) meaningful. I don't know how it compares to a hand-written extension.
Re: Python extensions should be lazy
#5Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. The key for optimizing a Python extension is to minimize the number of times you have to interact with Python. A couple of other tips in addition to what this article provides: 1. Object pooling is quite useful as it can significantly cut down on the number of allocations. 2. Be very care…
Why not go all the way and limit the times you have to interact with Python to zero ;)
Re: Python extensions should be lazy
#6Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. The key for optimizing a Python extension is to minimize the number of times you have to interact with Python. A couple of other tips in addition to what this article provides: 1. Object pooling is quite useful as it can significantly cut down on the number of allocations. 2. Be very care…
re: 3, Python has a native numeric array type https://docs.python.org/3/library/array.html
Re: Python extensions should be lazy
#7Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. The key for optimizing a Python extension is to minimize the number of times you have to interact with Python. A couple of other tips in addition to what this article provides: 1. Object pooling is quite useful as it can significantly cut down on the number of allocations. 2. Be very care…
Re: 2, is there any good repo with raw C Python API that can be used as a reference for someone who is not too proficient in C? I took a look at numpy but it seems too complicated for me
Re: Python extensions should be lazy
#8This is an impressive post showing some nice investigative work that isolates a pain point and produces a performant work-around.
However, the conclusion is debatable. Not everyone has this problem. Not everyone would benefit from the same solution.
Sure, if your data can be loaded, manipulated, and summarized outside of Python land, then lazy object creation is a good way to go. But then you're giving up all of the Python tooling that likely drove you to Python in the first place.
Most of the Python ecosystem from sets and dicts to the standard library is focused on manipulating native Python objects. While the syntax supports method calls to data encapsulated elsewhere, it can be costly to constantly "box and unbox" data to move back and forth between the two worlds.
Re: Python extensions should be lazy
#9Optimizing Python extensions is becoming increasingly important as Python is used in more and more compute intensive environments. The key for optimizing a Python extension is to minimize the number of times you have to interact with Python. A couple of other tips in addition to what this article provides: 1. Object pooling is quite useful as it can significantly cut down on the number of allocations. 2. Be very care…
re: 3, Python has a native numeric array type https://docs.python.org/3/library/array.html
We should probably get rid of that. It is old (predating numpy) and has limited functionality. In almost every case I can think of, you would be better off with numpy.
Re: Python extensions should be lazy
#10> In the case of ASTs, one could imagine a kind of ‘query language’ API for Python that operates on data that is owned by the extension - analogous to SQL over the highly specialized binary representations that a database would use. This would let the extension own the memory, and would lazily create Python objects when necessary.
You could make the API transparently lazy, i.e. ast.parse creates only one AstNode object or whatever and when you ask that object for e.g. its children those are created lazily from the underlying C struct. To preserve identity (which I assume is something users of ast are more likely to rely on than usual) you'd have to add some extra book-keeping to make it not generate new objects for each access, but memoize them.