No post body was provided.
Pyston-lite: our Python JIT as an extension module
11–20 of 35 posts
Re: Pyston-lite: our Python JIT as an extension module
#12Can someone ELI5 how this works as an extension module ? Is it possible to do a similar thing with pypy ?
I haven't read their source code, but the API extensions defined in PEP 523[1] lets extension modules (written in C or C level languages) to replace the default interpreter loop[2] with a custom one. I suspect they're using this mechanism to replace the default interpreter loop with their custom one when the module is imported and the enable function is called.
[1]: https://peps.python.org/pep-0523/ [2]: https://github.com/python/cpython/blob/main/Python/ceval.c
Re: Pyston-lite: our Python JIT as an extension module
#13[flagged]
Re: Pyston-lite: our Python JIT as an extension module
#14Earlier quoted context omitted.
Because it’s been decades, and the Python team has repeatedly made it clear that interpreter source simplicity is more important than performance. They’ve admitted it many times.
It might finally be changing, though! We'll have to see how the following releases go
Yes, it might be, but until them, some of us have to use options that exist.
Re: Pyston-lite: our Python JIT as an extension module
#15I love how easy this is to try out! I use pipenv, so I ran this: pipenv shell --python=python3.8 pip install pyston_lite_autoload And it worked: I got a small but material speed improvement from a tiny benchmark I ran against my own project: https://simonwillison.net/2022/Jun/8/pyston-lite/
My very unscientific test (just running the test suite on the Django project I happen to have up at the moment) proved to be very slightly slower with it installed: 11.2s - 11.3s without Pyston, 11.7s - 11.8s with Pyston.
Re: Pyston-lite: our Python JIT as an extension module
#16I love how easy this is to try out! I use pipenv, so I ran this: pipenv shell --python=python3.8 pip install pyston_lite_autoload And it worked: I got a small but material speed improvement from a tiny benchmark I ran against my own project: https://simonwillison.net/2022/Jun/8/pyston-lite/
My very unscientific test (just running the test suite on the Django project I happen to have up at the moment) proved to be very slightly slower with it installed: 11.2s - 11.3s without Pyston, 11.7s - 11.8s with Pyston.
A test suite would be designed to call every routine in a project a limited number of times. This means the JIT compiler will be compiling each routine, for it to be run a few times then thrown away.
Realistic loads might have the same routines run more intensely and might see more benefit from the JIT compiler.
Re: Pyston-lite: our Python JIT as an extension module
#17Can someone ELI5 how this works as an extension module ? Is it possible to do a similar thing with pypy ?
I haven't read their source code, but the API extensions defined in PEP 523[1] lets extension modules (written in C or C level languages) to replace the default interpreter loop[2] with a custom one. I suspect they're using this mechanism to replace the default interpreter loop with their custom one when the module is imported and the enable function is called. [1]: https://peps.python.org/pep-0523/ [2]: https://gith…
You inspired me to take a look at their code.
The autoload module at https://github.com/pyston/pyston/blob/96d5d33186b81f96ce3d9a... just calls:
__import__("pyston_lite").enable()
It calls jit_start() which I think is here and does more of the interesting work: https://github.com/pyston/pyston/blob/69b190003f14dfd2f6d276...
Re: Pyston-lite: our Python JIT as an extension module
#18I love how easy this is to try out! I use pipenv, so I ran this: pipenv shell --python=python3.8 pip install pyston_lite_autoload And it worked: I got a small but material speed improvement from a tiny benchmark I ran against my own project: https://simonwillison.net/2022/Jun/8/pyston-lite/
My very unscientific test (just running the test suite on the Django project I happen to have up at the moment) proved to be very slightly slower with it installed: 11.2s - 11.3s without Pyston, 11.7s - 11.8s with Pyston.
As someone else mentioned, test suites are a bit of a tough case for JITs. That said, we really don't want to slow down any workloads, and I suspect that something might be going wrong if you are getting a measurable slowdown.
Is this project public? I'd love to investigate
Re: Pyston-lite: our Python JIT as an extension module
#19Why not to just write entire project in cython?
Re: Pyston-lite: our Python JIT as an extension module
#20Any comparison with https://mypyc.readthedocs.io/en/latest/introduction.html ?