Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python 3.11 vs 3.10 performance

github.com

Post reply on HN
Print view View original

21–30 of 460 posts

Re: Python 3.11 vs 3.10 performance

#21

Post by Alex3917 »

uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

> Of course one wonders why they haven't been a low-hanging fruit over the past 25 years.

Because it's developed and maintained by volunteers, and there aren't enough folks who want to spend their volunteer time messing around with assembly language. Nor are there enough volunteers that it's practical to require very advanced knowledge of programming language design theory and compiler design theory as a prerequisite for contributing. People will do that stuff if they're being paid 500k a year, like the folks who work on v8 for Google, but there aren't enough people interested in doing it for free to guarantee that CPpython will be maintained in the future if it goes too far down that path.

Don't get me wrong, the fact that Python is a community-led language that's stewarded by a non-profit foundation is imho its single greatest asset. But that also comes with some tradeoffs.

Re: Python 3.11 vs 3.10 performance

#22

Post by BugsJustFindMe »

siekmanj wrote: post #6
uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

Python has actually had concurrency since about 2019: https://docs.python.org/3/library/asyncio.html . Having used it a few times, it seems fairly sane, but tbf my experience with concurrency in other languages is fairly limited. edit: ray https://github.com/ray-project/ray is also pretty easy to use and powerful for actual parallelism

I find asyncio to be horrendous, both because of the silliness of its demands on how you build your code and also because of its arbitrarily limited scope. Thread/ProcessPoolExecutor is personally much nicer to use and universally applicable...unless you need to accommodate Ctrl-C and then it's ugly again. But fixing _that_ stupid problem would have been a better expenditure of effort than asyncio.

Re: Python 3.11 vs 3.10 performance

#23

Post by fastball »

siekmanj wrote: post #6
uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

Python has actually had concurrency since about 2019: https://docs.python.org/3/library/asyncio.html . Having used it a few times, it seems fairly sane, but tbf my experience with concurrency in other languages is fairly limited. edit: ray https://github.com/ray-project/ray is also pretty easy to use and powerful for actual parallelism

asyncio is still single-threaded due to the GIL.

Re: Python 3.11 vs 3.10 performance

#24

Post by MarkSweep »

sod wrote: post #14

I wish there was something like llvm for scripting languages. Imagine if python, php, javascript, dart or ruby would not interpret the code themself, but compile to an interpretable common language, where you could just plug in the fastest interpreter there is for the job.

The RPython language PyPy is written in is designed to enable writing JIT-compiled interceptors for any language:

https://rpython.readthedocs.io/en/latest/

There is also the Dynamic Language Runtime, which is used to implement IronPython and IronRuby on top of .NET:

https://en.wikipedia.org/wiki/Dynamic_Language_Runtime

Re: Python 3.11 vs 3.10 performance

#25

Post by dagmx »

uniqueuid wrote: post #15
MrYellowP wrote: post #10

Earlier quoted context omitted.

I don't wonder. For me it seems pretty clear. I believe the reason is that python does not need any low-hanging fruits to have people use it, which is why they're a priority for so many other projects out there. Low-hanging fruits attract people who can't reach higher than that. When talking about low-hanging fruits, it's important to consider who they're for. The intended target audience. It's important to ask ones…

I see your point, but it directly conflicts with the effort many people put into producing extremely fast libraries for specific purposes, such as web frameworks (benchmarked extensively), ORMs and things like json and date parsing, as seen in the excellent ciso8601 [1] for example. [1] https://github.com/closeio/ciso8601

I disagree that it conflicts. There's an (implied) ceiling on Python performance, even after optimizations. The fear has always been that removing the design choices that cause that ceiling, would result in a different, incompatible language or runtime.

If everyone knows it's never going to reach the performance needed for high performance work, and there's already an excellent escape hatch in the form of C extensions, then why would people be spending time on the middle ground of performance? It'll still be too slow to do the things required, so people will still be going out to C for them.

Personally though, I'm glad for any performance increases. Python runs in so much critical infrastructure, that even a few percent would likely be a considerable energy savings when spread out over all users. Of course that assumes people upgrade their versions...but the community tends to be slow to do so in my experience.

Re: Python 3.11 vs 3.10 performance

#26

Post by TuringTest »

woodruffw wrote: post #16
tasubotadas wrote: post #8

Cool improvement but changes very little when Python is x100 times slower than other GC languages.

This is probably the wrong comparison to make: Python is two orders of magnitude slower than compiled GC languages, but it's in the same order of magnitude as most other interpreted GC languages. (It actually changes a whole lot, because there's a whole lot of code already out there written in Python. 25% faster is still 25% faster, even if the code would have been 100x faster to begin with in another language.)

> 25% faster is still 25% faster, even if the code would have been 100x faster to begin with in another language.)

And that's even assuming that the code would have existed at all in another language. The thing about interpreted GC languages is that the iterative loop of creation is much more agile, easier to start with and easier to prototype in than a compiled, strictly typed language.

Re: Python 3.11 vs 3.10 performance

#27

Post by int_19h »

uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

> one wonders why they haven't been a low-hanging fruit over the past 25 years.

From the very page you've linked to:

"Faster CPython explores optimizations for CPython. The main team is funded by Microsoft to work on this full-time. Pablo Galindo Salgado is also funded by Bloomberg LP to work on the project part-time."

Re: Python 3.11 vs 3.10 performance

#28

Post by speedgoose »

woodruffw wrote: post #16
tasubotadas wrote: post #8

Cool improvement but changes very little when Python is x100 times slower than other GC languages.

This is probably the wrong comparison to make: Python is two orders of magnitude slower than compiled GC languages, but it's in the same order of magnitude as most other interpreted GC languages. (It actually changes a whole lot, because there's a whole lot of code already out there written in Python. 25% faster is still 25% faster, even if the code would have been 100x faster to begin with in another language.)

Python being interpreted is the main reason why it’s slow, but not the excuse to not compare it to similar and faster programming languages.

Re: Python 3.11 vs 3.10 performance

#29

Post by pdpi »

MrYellowP wrote: post #10
uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

I don't wonder. For me it seems pretty clear. I believe the reason is that python does not need any low-hanging fruits to have people use it, which is why they're a priority for so many other projects out there. Low-hanging fruits attract people who can't reach higher than that. When talking about low-hanging fruits, it's important to consider who they're for. The intended target audience. It's important to ask ones…

> Low-hanging fruits attract people who can't reach higher than that.

Do we have completely different definitions of low-hanging fruit?

Python not "requiring" speed is a fair enough point if you want to argue against large complex performance-focused initiatives that consume too much of the team's time, but the whole point of calling something "low-hanging fruit" is precisely that they're easy wins — get the performance without a large effort commitment. Unless those easy wins hinder the language's core goals, there's no reason to portray it as good to actively avoid chasing those wins.

Re: Python 3.11 vs 3.10 performance

#30

Post by dagmx »

imwillofficial wrote: post #7

That’s wild, do we typically see such gains in dot releases? I don’t remember the last time this happened. Great news.

Programming language dot releases can tend to be significant.

Semver often means that major is the language version, and minor is the runtime version

Post reply on HN