High Performance Python Extensions: Part 1
crumpington.com
High Performance Python Extensions: Part 1
1–10 of 20 posts
Re: High Performance Python Extensions: Part 1
#2Why not Cython?
Re: High Performance Python Extensions: Part 1
#3Why not Cython?
My preference is to work in C directly. That being said, I may look into a Cython implementation for a future post.
Re: High Performance Python Extensions: Part 1
#4There are also other approaches like the HOPE jit https://github.com/cosmo-ethz/hope and Theano which is more about expression optimization and compilation: http://deeplearning.net/software/theano/
Re: High Performance Python Extensions: Part 1
#5If I understand correctly, the `pypy` people strongly encourage the use of `cffi` instead of the CPython API, as the latter is tied too much to CPython and does not permit efficient JITing.
Re: High Performance Python Extensions: Part 1
#6I recently cythonized the performance critical parts of a numpy/scipy based project with much success.
One does not necessarily need to get you hands dirty writing c-extensions ( although it can be a good exercise to learn the CPython API ).
In cython, you just need to sprinkle some static types to the inner loops and bump the speed up.
Re: High Performance Python Extensions: Part 1
#7Here's how I would write this, in Cython using "pure C" arrays:
https://gist.github.com/syllog1sm/3dd24cc8b0ad925325e1
It's getting 18,000 steps/second, in the same ballpark as your C code.
I prefer to "write C in Cython", because I find it easier to read than the numpy code. This may be my bias, though --- I've been writing almost nothing but Cython for about two years now.
Btw, if anyone's interested, "cymem" is a small library I have on pip. It's used to tie memory to a Python object's lifetime. All it does is remember what addresses it gave out, and when your Pool is garbage collected, it frees the memory.
Edit: GH fork, with code to compile and run the Cython version: https://github.com/syllog1sm/python-numpy-c-extension-exampl... . I hacked his script quickly.
Re: High Performance Python Extensions: Part 1
#8There are also other approaches like the HOPE jit https://github.com/cosmo-ethz/hope and Theano which is more about expression optimization and compilation: http://deeplearning.net/software/theano/
I wrote a blog post about HOPE http://blog.goranrakic.com/archives/2014/10/evaluating_hope....
Re: High Performance Python Extensions: Part 1
#9Here's how I would write this, in Cython using "pure C" arrays: https://gist.github.com/syllog1sm/3dd24cc8b0ad925325e1 It's getting 18,000 steps/second, in the same ballpark as your C code. I prefer to "write C in Cython", because I find it easier to read than the numpy code. This may be my bias, though --- I've been writing almost nothing but Cython for about two years now. Btw, if anyone's interested, "cymem" is a…
If I could have your permission, I'd like to incorporate this into a future post in the series. I can credit you in any way that you'd like.
Re: High Performance Python Extensions: Part 1
#10If I understand correctly, the `pypy` people strongly encourage the use of `cffi` instead of the CPython API, as the latter is tied too much to CPython and does not permit efficient JITing.
PyPy people also happily encourage to just use python and not rewrite anything to C (or not rewrite most stuff). Those loops should be really really fast on PyPy btw, written in pure python (with numpy arrays)