Pyston-lite: our Python JIT as an extension module

#1

Post by kmod » Wed, Jun 08, 2022, 4:48 PM UTC

Pyston-lite: our Python JIT as an extension module
blog.pyston.org

Re: Pyston-lite: our Python JIT as an extension module

#2

Post by metadat » Wed, Jun 08, 2022, 5:24 PM UTC

Pyston is neat, but I'm a bit unclear. Why should we invest time and effort into more proprietary shenanigans?

Re: Pyston-lite: our Python JIT as an extension module

#3

Post by nomel » Wed, Jun 08, 2022, 5:40 PM UTC

metadat wrote: post #2

Pyston is neat, but I'm a bit unclear. Why should we invest time and effort into more proprietary shenanigans?

Because it’s been decades, and the Python team has repeatedly made it clear that interpreter source simplicity is more important than performance. They’ve admitted it many times.

Re: Pyston-lite: our Python JIT as an extension module

#4

Post by colatkinson » Wed, Jun 08, 2022, 5:45 PM UTC

metadat wrote: post #2

Pyston is neat, but I'm a bit unclear. Why should we invest time and effort into more proprietary shenanigans?

Is it proprietary? According to their blog, Pyston was open sourced with version 2.2 [0], and the LICENSE file in the repo [1] appears to be the same as upstream CPython.

Not a rhetorical question BTW. A pluggable JIT for Python could be a boon for some projects at my dayjob, but if it's proprietary that would put a bit of a damper on things.

[0]: https://blog.pyston.org/2021/05/05/pyston-v2-2-faster-and-op...

[1]: https://github.com/pyston/pyston/blob/pyston_main/LICENSE

Re: Pyston-lite: our Python JIT as an extension module

#5

Post by make3 » Wed, Jun 08, 2022, 5:47 PM UTC

With the large performance increases brought by the more recent versions of Python, it's not clear to me that installing this is faster than just upgrading Python. It is more simple though I'll give you that. If it also faster with more recent versions?

Re: Pyston-lite: our Python JIT as an extension module

#6

Post by kzrdude » Wed, Jun 08, 2022, 5:52 PM UTC

nomel wrote: post #3

metadat wrote: post #2

Pyston is neat, but I'm a bit unclear. Why should we invest time and effort into more proprietary shenanigans?

Because it’s been decades, and the Python team has repeatedly made it clear that interpreter source simplicity is more important than performance. They’ve admitted it many times.

It might finally be changing, though! We'll have to see how the following releases go

Re: Pyston-lite: our Python JIT as an extension module

#7

Post by metadat » Wed, Jun 08, 2022, 6:01 PM UTC

colatkinson wrote: post #4

metadat wrote: post #2

Pyston is neat, but I'm a bit unclear. Why should we invest time and effort into more proprietary shenanigans?

Is it proprietary? According to their blog, Pyston was open sourced with version 2.2 [0], and the LICENSE file in the repo [1] appears to be the same as upstream CPython. Not a rhetorical question BTW. A pluggable JIT for Python could be a boon for some projects at my dayjob, but if it's proprietary that would put a bit of a damper on things. [0]: https://blog.pyston.org/2021/05/05/pyston-v2-2-faster-and-op... [1]: h…

Thanks, I didn't know Pyston had been de-proprietarized and open sourced.

Re: Pyston-lite: our Python JIT as an extension module

#8

Post by simonw » Wed, Jun 08, 2022, 6:03 PM UTC

I love how easy this is to try out!

I use pipenv, so I ran this:

    pipenv shell --python=python3.8
    pip install pyston_lite_autoload

And it worked: I got a small but material speed improvement from a tiny benchmark I ran against my own project: https://simonwillison.net/2022/Jun/8/pyston-lite/

Re: Pyston-lite: our Python JIT as an extension module

#9

Post by vsskanth » Wed, Jun 08, 2022, 6:15 PM UTC

Can someone ELI5 how this works as an extension module ? Is it possible to do a similar thing with pypy ?

Re: Pyston-lite: our Python JIT as an extension module

#10

Post by jorl17 » Wed, Jun 08, 2022, 6:21 PM UTC

make3 wrote: post #5

With the large performance increases brought by the more recent versions of Python, it's not clear to me that installing this is faster than just upgrading Python. It is more simple though I'll give you that. If it also faster with more recent versions?

I believe the article answers all your questions!

(Also, IMO, it is much easier to install Pyston-lite than upgrade a whole Python version, which is an absolute nightmare on some of my specific scenarios with somewhat embedded hardware)

They are currently stuck on 3.8, but believe they will be able to support multiple versions easily because pyston-lite is much easier to develop for than pyston (because it is an extension and not a whole CPython fork).

They also present benchmarks at the very end comparing Pyston, Pyston-lite and the latest CPython 3.11.0b3, all compared to Ubuntu's default Python 3.8.10. It shows that while CPython offers ~8-15% improvements (macro-micro benchmarks), Pyston-Lite offers 8-27% improvements and "OG" Pyston offers 35-65% improvements.

So compared to regular CPython you might get an extra 10% or so, which might or not be worth it depending on your case. Excited to see what happens if they port Pyston to Python 3.11, though!