Python 3.11 vs 3.10 performance • Hacker News Board

Re: Python 3.11 vs 3.10 performance

#241

Post by wheelerof4te » Wed, Jul 06, 2022, 7:05 PM UTC

superjan wrote: post #198

Yesterday, I watched Emery Berger’s “Python performance matters” Which is you should not bother to write fast Python, just delegate all heavy lifting to optimized C/c++ and the likes. His group has a Python profiler whose main goal is to point out which parts should be delegated. Of course optimized is better but the numeric microbenchmarks in this post are mostly examples of code that is better delegated to low over…

The key wording is "optimized C".

CPython is already executed in pure C, the only difference it being a very slow, unoptimized C.

The code for simple adding of two numbers is insane. It jumps through enough hoops to make you wonder how is it even running everything else.

Re: Python 3.11 vs 3.10 performance

#242

Post by mixmastamyk » Wed, Jul 06, 2022, 7:06 PM UTC

coliveira wrote: post #163

ianbicking wrote: post #82

Earlier quoted context omitted.

Python is compiled to a VM (that is CPython). The semantics and implementation of the VM haven't prioritized performance as much as some other systems. Here we're seeing improvements in the implementation. But the semantics will be the hard part, as those semantics limit the performance. For instance "a + b" is (I believe) compiled into bytecodes that pretty much follow the expression. But the implementation of "add"…

Dynamic languages like Javascript have solved this problem by essentially caching the resolution of a particular expression that is executed very often. I don't see why this cannot be done in Python.

The expressions are dynamic, so they have to be evaluated every time.

Python is excessively dynamic, so it can't (conventionally) be sped up as easily as many other languages unfortunately. Finally some folks are being paid and allowed to be working on it.

Re: Python 3.11 vs 3.10 performance

#243

Post by lrtap » Wed, Jul 06, 2022, 7:08 PM UTC

antman wrote: post #208

There was always a denial of removing the Global Interpreter Lock because it would decrease single threaded Python speed for which most people din’t care. So I remember a guy recently came up with a patch that both removed GIL and also to make it easier for the core team to accept it he added also an equivalent number of optimizations. I hope this release was is not we got the optimizations but ignored the GIL part.…

> I hope this release was is not we got the optimizations but ignored the GIL part.

I would not be surprised. It is highly likely that the optimizations will be taken, credit will go to the usual people and the GIL part will be extinguished.

Re: Python 3.11 vs 3.10 performance

#244

Post by cercatrova » Wed, Jul 06, 2022, 7:08 PM UTC

objektif wrote: post #238

cercatrova wrote: post #234

Somewhat related, had anyone used Nim? These days I've been using it for its Python like syntax year C like speed, especially if I just need to script something quickly.

Is it worth it to learn? Especially when you have python?

Yeah it's a nice language, it has optional static types as well which is always a plus in my experience. The speed though is what's really interesting.

Re: Python 3.11 vs 3.10 performance

#245

Post by wheelerof4te » Wed, Jul 06, 2022, 7:11 PM UTC

polski-g wrote: post #173

teekert wrote: post #161

Earlier quoted context omitted.

Whenever I nest a list comprehension I feel dirty anyway, I wonder if a better error message will help solve something that is just hard to grasp sometimes.

All list comprehensions should error with message "FATAL PARSE ERROR: Will you really understand this in six months?"

List comprehensions should be used for making quick lists that contain simple calculations or filters.

Anything more will come back to bite you later.

Re: Python 3.11 vs 3.10 performance

#246

Post by wvh » Wed, Jul 06, 2022, 7:12 PM UTC

yantrams wrote: post #199

agumonkey wrote: post #175

Earlier quoted context omitted.

In a way it's a nice incentive not to nest too much (I have that tendency too)

Hold my beer! https://i.redd.it/8waggyjyyle51.png

I don't think this is bad code. Maybe not the most Pythonic in the imperative sense, but certainly you could come across such constructs in languages that are more oriented towards functional programming. This could also be solved in a more readable manner by having a generator pipeline, though it would be a good idea to see what the performance of chained generators is like in your version and flavour of Python.

Re: Python 3.11 vs 3.10 performance

#247

Post by SiempreViernes » Wed, Jul 06, 2022, 7:19 PM UTC

metadat wrote: post #237

BerislavLopac wrote: post #229

Earlier quoted context omitted.

The GIL removal by that guy reverted some of the improvement done by other optimisations, so the overall improvement was much smaller. And most people do care for single-threaded speed, because the vast majority of Python software is written as single-threaded.

> the vast majority of Python software is written as single-threaded. This is a self-fulfilling prophecy, as the GIL makes Python's (and Ruby's) concurrency story pretty rough compared to nearly all other widely used languages: C, C++, Java, Go, Rust, and even Javascript (as of late).

Pretty sure it is actually a statement that comes from the "python is a scripting language" school, and not because the huge horde of programmers that craves concurrency when they write a script to reformat a log file to csv keeps being put off by the python multiprocessing story.

Re: Python 3.11 vs 3.10 performance

#248

Post by metadat » Wed, Jul 06, 2022, 7:22 PM UTC

SiempreViernes wrote: post #247

metadat wrote: post #237

Earlier quoted context omitted.

> the vast majority of Python software is written as single-threaded. This is a self-fulfilling prophecy, as the GIL makes Python's (and Ruby's) concurrency story pretty rough compared to nearly all other widely used languages: C, C++, Java, Go, Rust, and even Javascript (as of late).

Pretty sure it is actually a statement that comes from the "python is a scripting language" school, and not because the huge horde of programmers that craves concurrency when they write a script to reformat a log file to csv keeps being put off by the python multiprocessing story.

Not sure I understand your point, can you clarify? Python is used across many different domains, being able to really take advantage of multiple cores would be a big deal.

I'd really appreciate if python included concurrency or parallelism capabilities that didn't disappoint and frustrate me.

If you've tried using the thread module, multiprocessing module, or async function coloring feature, you probably can relate. They can sort of work but are about as appealing as being probed internally in a medical setting.

Re: Python 3.11 vs 3.10 performance

#249

Post by dekhn » Wed, Jul 06, 2022, 7:22 PM UTC

tnvaught wrote: post #236

dekhn wrote: post #37

Earlier quoted context omitted.

Not really. There were a couple engineers working at Google on a project called unladen swallow which was extremely promising but it eventually got canceled. The developer who worked at Microsoft to make iron python I think that was his full-time project as well and it was definitely faster than cpython at the time

"The developer who worked at Microsoft to make iron python" - Jim Hugunin [1] (say his name!) to whom much is owed by the python community and humanity, generally. [1] https://en.wikipedia.org/wiki/Jim_Hugunin

I'm sorry I had forgotten at the time. I almost got to work with him at Google. I agree he's a net positive for humanity (I used numeric heavily back in the day)

Re: Python 3.11 vs 3.10 performance

#250

Post by tyingq » Wed, Jul 06, 2022, 7:24 PM UTC

metadat wrote: post #237

BerislavLopac wrote: post #229

Earlier quoted context omitted.

The GIL removal by that guy reverted some of the improvement done by other optimisations, so the overall improvement was much smaller. And most people do care for single-threaded speed, because the vast majority of Python software is written as single-threaded.

> the vast majority of Python software is written as single-threaded. This is a self-fulfilling prophecy, as the GIL makes Python's (and Ruby's) concurrency story pretty rough compared to nearly all other widely used languages: C, C++, Java, Go, Rust, and even Javascript (as of late).

Getting rid of the GIL will also immediately expose all the not-thread-safe stuff that currently exists, so there's a couple of waves you would need before it would be broadly usable.