See also upcoming Python 3.11 performance improvements
Pyston-lite: our Python JIT as an extension module
21–30 of 35 posts
Re: Pyston-lite: our Python JIT as an extension module
#22See also upcoming Python 3.11 performance improvements https://news.ycombinator.com/item?id=31642793
Pyston is substantially faster for our internal benchmarks than Py3.11
The only downside is it’s a 3.8 fork, which fortunately isn’t an issue for us.
Re: Pyston-lite: our Python JIT as an extension module
#23Why not to just write entire project in cython?
[deleted]
Re: Pyston-lite: our Python JIT as an extension module
#24Can someone ELI5 how this works as an extension module ? Is it possible to do a similar thing with pypy ?
> Is it possible to do a similar thing with pypy ?
The predecessor of Pypy is called Psyco http://psyco.sourceforge.net/
To enable JIT in Python2.x, simply `import psyco; psyco.full()`
It was fun while it lasted.
For Pypy3, check out https://github.com/fijal/jitpy
Re: Pyston-lite: our Python JIT as an extension module
#25Earlier quoted context omitted.
My very unscientific test (just running the test suite on the Django project I happen to have up at the moment) proved to be very slightly slower with it installed: 11.2s - 11.3s without Pyston, 11.7s - 11.8s with Pyston.
A test suite would be designed to call every routine in a project a limited number of times. This means the JIT compiler will be compiling each routine, for it to be run a few times then thrown away. Realistic loads might have the same routines run more intensely and might see more benefit from the JIT compiler.
Ahhh, of course. Good point, thanks.
Re: Pyston-lite: our Python JIT as an extension module
#26Earlier quoted context omitted.
My very unscientific test (just running the test suite on the Django project I happen to have up at the moment) proved to be very slightly slower with it installed: 11.2s - 11.3s without Pyston, 11.7s - 11.8s with Pyston.
As someone else mentioned, test suites are a bit of a tough case for JITs. That said, we really don't want to slow down any workloads, and I suspect that something might be going wrong if you are getting a measurable slowdown. Is this project public? I'd love to investigate
It's not, but after reading the other answer here I can totally see why it might be slightly slower. I'll definitely do some more testing with it beyond just timing my test suite runs.
Re: Pyston-lite: our Python JIT as an extension module
#27Why not to just write entire project in cython?
Why not just write entire project in (pick a language).
Sometimes you have a Python project already, you aren't planning to write a new project. In fact there's a ton of non cython python in existence, every time I use a new library, your solution is to rewrite it in cython?
Lots of ways to speed up python. It's great to have this option, which will work for some without having to change much of anything. Cython, Numba, C (etc.) extensions, ... All are good, too, where they fit into a project and development cycle.
Re: Pyston-lite: our Python JIT as an extension module
#28Is there a reason for the use of system() here?
https://github.com/pyston/pyston/blob/69b190003f14dfd2f6d276...
Seems easier to use the C functions to do this, rather than rely on system commands.
Re: Pyston-lite: our Python JIT as an extension module
#29Any comparison with https://mypyc.readthedocs.io/en/latest/introduction.html ?
The main difference is that Pyston-Lite is a CPython plugin, while MyPyC is an AOT compiler that emits CPython extension modules.
But if you're asking about performance and use cases, there are a lot of Python compilers and alternative implementations out there now!
Not just MyPyC, but also Nuitka, Shedskin (not sure if still in development), GraalPython, PyPy, Cinder, Pyjion, IronPython (not sure if still in development), and even Stackless Python (yes it's still in develoment). Not to mention Cython and, for specific numerical tasks, Numba.
I think Microsoft also recently announced a CPython JIT extension that's analogous to Pyston-Lite.
A benchmark or comprehensive comparison suite would be pretty cool, but I'm not aware of one. Personally I think CPython plugins are the most user-friendly option, because they require the fewest changes in your deployment setup and runtime environment. PyPy isn't that bad but isn't perfect either. Whereas ahead-of-time compilation is a pretty big change from the usual Python developer workflow and might not be easy to convince a team to adopt it.
Re: Pyston-lite: our Python JIT as an extension module
#30Pyston is neat, but I'm a bit unclear. Why should we invest time and effort into more proprietary shenanigans?
Is it proprietary? According to their blog, Pyston was open sourced with version 2.2 [0], and the LICENSE file in the repo [1] appears to be the same as upstream CPython. Not a rhetorical question BTW. A pluggable JIT for Python could be a boon for some projects at my dayjob, but if it's proprietary that would put a bit of a damper on things. [0]: https://blog.pyston.org/2021/05/05/pyston-v2-2-faster-and-op... [1]: h…
This is a drop-in replacement for the CPython bytecode interpreter, right?
Didn't Microsoft also recently announce something similar? I could have sworn I saw a thread about it here recently, but i couldn't find it again. Or was that the Facebook project Cinder?
There are so many "high performance Python" projects now (and in the past) that it's hard to keep track!