Python 3.11 vs 3.10 performance • Hacker News Board

Re: Python 3.11 vs 3.10 performance

#191

Post by LtWorf » Wed, Jul 06, 2022, 5:50 PM UTC

dekhn wrote: post #139

dragonwriter wrote: post #134

Earlier quoted context omitted.

That's absolutely not the case with IronPython, where being tied to the .NET ecosystem was very much the point. But if it was just core-team resistance and not a more fundamentally different objective than just improving the core interpreter performance, maintaining support for native extensions while speeding up the implementation would have been a goal even if it had to be in a fork. They were solving a fundamental…

If I remember correctly iron python ran on mono it just didn't have all of the .net bits. I remember at the python conference when it was introduced he actually showed how you could bring up clippy or a wizard programmatically from python talking directly to the operating system through its native apis. Really the value came from the thread safe container/collections data types which are the foundation of high perfor…

Too bad .net is designed for windows and on linux the APIs feel like windows API that kinda sorta work on linux.

Re: Python 3.11 vs 3.10 performance

#192

Post by thesuperbigfrog » Wed, Jul 06, 2022, 5:51 PM UTC

ihaveajob wrote: post #143

gostsamo wrote: post #68

Earlier quoted context omitted.

A few people tried to excuse the slow python, but as far as I know the story, excuses are not necessary. Truth is that python was not meant to be fast, its source code was not meant to be fast, and its design was not optimized with the idea of being fast. Python was meant as a scrypting language that was easy to learn and work with on all levels and the issue of its slowness became important when it outgrew its role…

Indeed, and that's why I love it. When I need extra performance, which is very rarely, I don't mind spending extra effort outsourcing it to a binary module. More often, the problem is an inefficient algorithm or data arrangement. I took a graduate level data structures class and the professor used Python among other things "because it's about 80 times slower than C, so you have to think hard about your algorithms". A…

Global Interpreter Lock (GIL) is also an issue affecting Python execution speed:

https://en.wikipedia.org/wiki/Global_interpreter_lock

Re: Python 3.11 vs 3.10 performance

#193

Post by LtWorf » Wed, Jul 06, 2022, 5:52 PM UTC

simias wrote: post #52

uniqueuid wrote: post #2

These speedups are awesome, but of course one wonders why they haven't been a low-hanging fruit over the past 25 years. Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11. Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different lang…

I suspect that the amount of people and especially companies willing to spend time and money optimizing Python are fairly low. Think about it: if you have some Python application that's having performance issues you can either dig into a foreign codebase to see if you can find something to optimize (with no guarantee of result) and if you do get something done you'll have to get the patch upstream. And all that "only…

There was some guarantee of result. It has been a long process but there was mostly one person who had identified a number of ways to make it faster but wanted financing to actually do the job. Seems Microsoft is doing the financing, but this has been going on for quite a while.

Re: Python 3.11 vs 3.10 performance

#194

Post by timoteostewart » Wed, Jul 06, 2022, 5:55 PM UTC

8organicbits wrote: post #185

teekert wrote: post #161

Earlier quoted context omitted.

Whenever I nest a list comprehension I feel dirty anyway, I wonder if a better error message will help solve something that is just hard to grasp sometimes.

"Debugging is twice as hard as writing the code in the first place. Therfore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Rajanand Although, personally, I enjoy python list comprehensions.

I believe that quote about debugging being twice as hard as coding is by Brian Kernighan.

https://en.m.wikiquote.org/wiki/Brian_Kernighan

Re: Python 3.11 vs 3.10 performance

#195

Post by [deleted] » Wed, Jul 06, 2022, 5:56 PM UTC

ChuckNorris89 wrote: post #57

int_19h wrote: post #27

Earlier quoted context omitted.

> one wonders why they haven't been a low-hanging fruit over the past 25 years. From the very page you've linked to: "Faster CPython explores optimizations for CPython. The main team is funded by Microsoft to work on this full-time. Pablo Galindo Salgado is also funded by Bloomberg LP to work on the project part-time."

Hmm, looks like corporate money gets sh*t done.

[deleted]

Re: Python 3.11 vs 3.10 performance

#196

Post by wzdd » Wed, Jul 06, 2022, 5:58 PM UTC

winstonewert wrote: post #108

GrayShade wrote: post #102

Earlier quoted context omitted.

You should probably read the full context around that quote, I'm sick and tired of everyone repeating it mindlessly: https://softwareengineering.stackexchange.com/a/80092 > Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.

> You should probably read the full context around that quote, I'm sick and tired of everyone repeating it mindlessly: I'm confused. What do you think the context changes? At least as I read it, both the short form and full context convey the same idea.

The quote at least contextualises "premature". As it is, premature optimisation is by definition inappropriate -- that's what "premature" means. The context:

a) gives a rule-of-thumb estimate of how much optimisation to do (maybe 3% of all opportunities);

b) explains that non-premature opimisation is not just not the root of all evil but actually a good thing to do; and

c) gives some information about how to do non-premature optimisation, by carefully identifying performance bottlenecks after the unoptimised code has been written.

I agree with GP that unless we know what Knuth meant by "premature" it is tempting to use this quote to justify too little optimisation.

Re: Python 3.11 vs 3.10 performance

#197

Post by vorticalbox » Wed, Jul 06, 2022, 5:59 PM UTC

BeetleB wrote: post #147

atwood22 wrote: post #79

Earlier quoted context omitted.

> Python was meant as a scrypting language that was easy to learn and work with on all levels. Being fast isn't contradictory with this goal. If anything, this is a lesson that so many developers forget. Things should be fast by default.

> Things should be fast by default. In over 90% of my work in the SW industry, being fast(er) was of no benefit to anyone. So no, it should not be fast by default.

Being first to market is usually more important than being faster then those who got there first.

Re: Python 3.11 vs 3.10 performance

#198

Post by superjan » Wed, Jul 06, 2022, 6:00 PM UTC

Yesterday, I watched Emery Berger’s “Python performance matters” Which is you should not bother to write fast Python, just delegate all heavy lifting to optimized C/c++ and the likes. His group has a Python profiler whose main goal is to point out which parts should be delegated. Of course optimized is better but the numeric microbenchmarks in this post are mostly examples of code that is better delegated to low overhead compiled languages.

https://youtu.be/LLx428PsoXw

Re: Python 3.11 vs 3.10 performance

#199

Post by yantrams » Wed, Jul 06, 2022, 6:04 PM UTC

agumonkey wrote: post #175

bilsbie wrote: post #55

Earlier quoted context omitted.

When is that coming? Yes the error messages are confusing in a big list comprehension.

In a way it's a nice incentive not to nest too much (I have that tendency too)

Hold my beer!

https://i.redd.it/8waggyjyyle51.png

Re: Python 3.11 vs 3.10 performance

#200

Post by uoaei » Wed, Jul 06, 2022, 6:05 PM UTC

BeetleB wrote: post #147

atwood22 wrote: post #79

Earlier quoted context omitted.

> Python was meant as a scrypting language that was easy to learn and work with on all levels. Being fast isn't contradictory with this goal. If anything, this is a lesson that so many developers forget. Things should be fast by default.

> Things should be fast by default. In over 90% of my work in the SW industry, being fast(er) was of no benefit to anyone. So no, it should not be fast by default.

Ha! Good luck convincing end-users that that's true.