Python 3.11 vs 3.10 performance • Hacker News Board

Re: Python 3.11 vs 3.10 performance

#71

Post by dekhn » Wed, Jul 06, 2022, 3:53 PM UTC

pstrateman wrote: post #63

dekhn wrote: post #37

Earlier quoted context omitted.

Not really. There were a couple engineers working at Google on a project called unladen swallow which was extremely promising but it eventually got canceled. The developer who worked at Microsoft to make iron python I think that was his full-time project as well and it was definitely faster than cpython at the time

Neither of those projects were ever going to be accepted upstream though.

My hope was that iron pythonreplaced cpython as the standard but for a number of reasons that was never to be.

Re: Python 3.11 vs 3.10 performance

#72

Post by kevin_thibedeau » Wed, Jul 06, 2022, 3:53 PM UTC

speedgoose wrote: post #28

woodruffw wrote: post #16

Earlier quoted context omitted.

This is probably the wrong comparison to make: Python is two orders of magnitude slower than compiled GC languages, but it's in the same order of magnitude as most other interpreted GC languages. (It actually changes a whole lot, because there's a whole lot of code already out there written in Python. 25% faster is still 25% faster, even if the code would have been 100x faster to begin with in another language.)

Python being interpreted is the main reason why it’s slow, but not the excuse to not compare it to similar and faster programming languages.

You can completely bypass the interpreter by running plain, unannotated Python code through Cython. You get a speedup but it is still slow from manipulating PyObjects and attribute lookups.

Re: Python 3.11 vs 3.10 performance

#73

Post by dragonwriter » Wed, Jul 06, 2022, 3:54 PM UTC

samuelstros wrote: post #35

sod wrote: post #14

I wish there was something like llvm for scripting languages. Imagine if python, php, javascript, dart or ruby would not interpret the code themself, but compile to an interpretable common language, where you could just plug in the fastest interpreter there is for the job.

Is that not where WebAssembly is headed? Taken from their website "Wasm is designed as a portable compilation target for programming languages [...]"

Not really. Scripting languages implemented on WASM are doing it by compiling their main interpreter to WASM, not the source that the interpreter would execute.

Re: Python 3.11 vs 3.10 performance

#74

Post by oaiey » Wed, Jul 06, 2022, 3:54 PM UTC

uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

There is a race right now to be more performant. .NET, Java, Go already participate, Rust/C++ are anyway there. So to stay relevant they have to start participate. .NET went through the same some years ago.

And why they were not addressed: because starting a certain points, the optimization are e.g. processor specific or non intuitive to understand. Making it hard to maintain vs a simple straight forward solution.

Re: Python 3.11 vs 3.10 performance

#75

Post by switch007 » Wed, Jul 06, 2022, 3:54 PM UTC

dekhn wrote: post #37

KptMarchewa wrote: post #17

Earlier quoted context omitted.

Answer is very simple. Amount of people who got paid to make python fast was rounded to 0.

Not really. There were a couple engineers working at Google on a project called unladen swallow which was extremely promising but it eventually got canceled. The developer who worked at Microsoft to make iron python I think that was his full-time project as well and it was definitely faster than cpython at the time

Was the project run by a team based in Africa or Europe?

Re: Python 3.11 vs 3.10 performance

#76

Post by staticassertion » Wed, Jul 06, 2022, 3:54 PM UTC

Alex3917 wrote: post #21

uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

> Of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Because it's developed and maintained by volunteers, and there aren't enough folks who want to spend their volunteer time messing around with assembly language. Nor are there enough volunteers that it's practical to require very advanced knowledge of programming language design theory and compiler design theory as a prerequisite…

> Because it's developed and maintained by volunteers

Who is working on python voluntarily? I would assume that, like the Linux kernel, the main contributors are highly paid. Certainly, having worked at Dropbox, I can attest to at least some of them being highly paid.

Re: Python 3.11 vs 3.10 performance

#77

Post by tambourine_man » Wed, Jul 06, 2022, 3:54 PM UTC

uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

That’s something that has always puzzled me as well.

Given the popularity, you’d think Python would have had several generations of JITs by now and yet it still runs interpreted, AFAIK.

JavaScript has proven that any language can be made fast given enough money and brains, no matter how dynamic.

Maybe Python's C escape hatch is so good that it’s not worth the trouble. It’s still puzzling to me though.

Re: Python 3.11 vs 3.10 performance

#78

Post by ikinsey » Wed, Jul 06, 2022, 3:54 PM UTC

fastball wrote: post #23

siekmanj wrote: post #6

Earlier quoted context omitted.

Python has actually had concurrency since about 2019: https://docs.python.org/3/library/asyncio.html . Having used it a few times, it seems fairly sane, but tbf my experience with concurrency in other languages is fairly limited. edit: ray https://github.com/ray-project/ray is also pretty easy to use and powerful for actual parallelism

asyncio is still single-threaded due to the GIL.

While not ideal, this can be mitigated with multiprocessing. Python asyncio exposes interfaces for interacting with multiple processes [1].

[1] https://docs.python.org/3/library/asyncio-eventloop.html#asy...

Re: Python 3.11 vs 3.10 performance

#79

Post by atwood22 » Wed, Jul 06, 2022, 3:55 PM UTC

gostsamo wrote: post #68

uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

A few people tried to excuse the slow python, but as far as I know the story, excuses are not necessary. Truth is that python was not meant to be fast, its source code was not meant to be fast, and its design was not optimized with the idea of being fast. Python was meant as a scrypting language that was easy to learn and work with on all levels and the issue of its slowness became important when it outgrew its role…

> Python was meant as a scrypting language that was easy to learn and work with on all levels.

Being fast isn't contradictory with this goal. If anything, this is a lesson that so many developers forget. Things should be fast by default.

Re: Python 3.11 vs 3.10 performance

#80

Post by upupandup » Wed, Jul 06, 2022, 3:55 PM UTC

bilsbie wrote: post #53

I’d be curious to see the speed up compared to python 2.4. To see how far it’s come since I first got started with it.

it means you are not following PEP8

if you get non descriptive errors, it means you haven't followed proper exception handling/management

its not the fault of Python but the developer. go ahead and downvote me but you if you mentioned parent's comment in an interview, you would not receive a call back or at least I hope the interviewer is realizing the skill gap.