Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python 3.11 vs 3.10 performance

github.com

Post reply on HN
Print view View original

261–270 of 460 posts

Re: Python 3.11 vs 3.10 performance

#261

Post by int_19h »

antman wrote: post #208

There was always a denial of removing the Global Interpreter Lock because it would decrease single threaded Python speed for which most people din’t care. So I remember a guy recently came up with a patch that both removed GIL and also to make it easier for the core team to accept it he added also an equivalent number of optimizations. I hope this release was is not we got the optimizations but ignored the GIL part.…

Removing GIL also breaks existing native packages, and would require wholesale migration across the entire ecosystem, on a scale not dissimilar to what we've seen with Python 3.

Re: Python 3.11 vs 3.10 performance

#262

Post by agumonkey »

ChadNauseam wrote: post #223
yantrams wrote: post #199

Earlier quoted context omitted.

Hold my beer! https://i.redd.it/8waggyjyyle51.png

Hold mine :D https://github.com/anchpop/genomics_viz/blob/master/genomics... That's one expression because it used to be part of a giant comprehension, but I moved it into a function for a bit more readability. I'm considering moving it back just for kicks though. My philosophy is: if you're only barely smart enough to code it, you aren't smart enough to debug it. Therefore, you should code at your limit, to force yo…

That code is fine in my book, there's more static code than for-comp and it's at work 3 level deep with no hard coupling between levels. Also well spaced and worded.

Re: Python 3.11 vs 3.10 performance

#263

Post by ModernMech »

superjan wrote: post #198

Yesterday, I watched Emery Berger’s “Python performance matters” Which is you should not bother to write fast Python, just delegate all heavy lifting to optimized C/c++ and the likes. His group has a Python profiler whose main goal is to point out which parts should be delegated. Of course optimized is better but the numeric microbenchmarks in this post are mostly examples of code that is better delegated to low over…

> you should not bother to write fast Python, just delegate all heavy lifting to optimized C/c++ and the likes

Certainly that's something you can do, but unfortunately for Python it opens the door for languages like Julia, which are trying to say that you can have your cake and eat it too.

Re: Python 3.11 vs 3.10 performance

#264

Post by zitterbewegung »

lanstin wrote: post #215
zitterbewegung wrote: post #136

Earlier quoted context omitted.

Back when Python was started there was really C or C++ for optimized programs and scripting languages like Python and Perl. But since Python had the ability for C Extensions it allowed it to bypass those problems. Since Python was easy to learn both web developers and scientists to learn. Then financial organizations started to get interested and that’s really how Python cemented itself. What exactly do you do with P…

I worked on Python almost exclusively for maybe five years. Then I tried go. Each time I wrote a go program, I am giddy with excitement at how fast my first attempt is, scaling so smoothly with the number of cores. I also wrote a lot of fairly performant C in the 90s, so I know what computers can do in a second. I still use Python for cases when dev time is more important than execution time (which is rarer now that…

Okay you are using Python gluing two web services together which is what you deep acceptable that Python can do but can you just comment on the things that you don't use Python anymore due to it being slow?

Don't take this the wrong way but I think you could be more specific. Are you saying that similar to Go it should be just faster in general?

Re: Python 3.11 vs 3.10 performance

#266

Post by int_19h »

ChadNauseam wrote: post #223
yantrams wrote: post #199

Earlier quoted context omitted.

Hold my beer! https://i.redd.it/8waggyjyyle51.png

Hold mine :D https://github.com/anchpop/genomics_viz/blob/master/genomics... That's one expression because it used to be part of a giant comprehension, but I moved it into a function for a bit more readability. I'm considering moving it back just for kicks though. My philosophy is: if you're only barely smart enough to code it, you aren't smart enough to debug it. Therefore, you should code at your limit, to force yo…

Yours is nice and readable. Parents' one is not, but it feels like the indentation is deliberately confusing. I'd lay it out like so:

   tags = list(set([
     nel
     for subli in [
       mel
       for subl in [
         [[jel.split('/')[2:] for jel in el] 
         for el in classified
       ]
       for mel in subl 
     ]
     for nel in subli
     if nel
   ]))
Still not very readable, tho. But that's largely due to Python's outputs-first sequence comprehension syntax being a mess that doesn't scale at all.

Side note: one other thing I always hated about those things in Python is that there's no way to bind an intermediary computation to a variable. In C# LINQ, you can do things like:

   from x in xs
   let y = x.ComputeSomething()
   where y.IsFoo && y.IsBar
   select y
In Python, you have to either invoke ComputeSomething twice, or hack "for" to work like "let" by wrapping the bound value in a container:

   y
   for x in xs
   for y in [x.ComputeSomething()]
   if y.IsFoo and y.IsBar
It's not just about not repeating yourself or not running the same code twice, either - named variables are themselves a form of self-documenting code, and a sequence pipeline using them even where they aren't strictly needed can be much more readable.

Re: Python 3.11 vs 3.10 performance

#267

Post by maxbond »

dragonwriter wrote: post #115
maxbond wrote: post #109

Earlier quoted context omitted.

This certainly does not mean, "tolerate absurd levels of technical debt, and only ever think about performance in retrospect."

Python was meeting needs well enough to be one of, if not the single, most popular language for a considerable time and continuing to expand and become dominant in new application domains while languages that focussed more heavily on performance rose and fell. And it's got commercial interests willing to throw money at performance now because of that. Seems like the Python community, whether as top-down strategy or e…

Python had strengths that drove it's adoption, namely that it introduced new ideas about a language's accessibility and readability. I'm not sure it was ever really meeting the needs of application developers. People have been upset about Python performance and how painful it is to write concurrent code for a long time. The innovations in accessibility and readability have been recognized as valuable - and adopted by other languages (Go comes to mind). More recently, it seems like Python is playing catch-up, bringing in innovations from other languages that have become the norm, such as asyncio, typing, even match statements.

Languages don't succeed on their technical merit. They succeed by being good enough to gain traction, after which it is more about market forces. People choose Python for it's great ecosystem and the availability of developers, and they accept the price they pay in performance. But that doesn't imply that performance wasn't an issue in the past, or that Python couldn't have been even more successful if it had been more performant.

And to be clear, I use Python every day, and I deeply appreciate the work that's been put into 3.10 and 3.11, as well as the decades prior. I'm not interested in prosecuting the decisions about priorities that were made in the past. But I do think there are lessons to be learned there.

Re: Python 3.11 vs 3.10 performance

#269

Post by smcl »

int_19h wrote: post #261
antman wrote: post #208

There was always a denial of removing the Global Interpreter Lock because it would decrease single threaded Python speed for which most people din’t care. So I remember a guy recently came up with a patch that both removed GIL and also to make it easier for the core team to accept it he added also an equivalent number of optimizations. I hope this release was is not we got the optimizations but ignored the GIL part.…

Removing GIL also breaks existing native packages, and would require wholesale migration across the entire ecosystem, on a scale not dissimilar to what we've seen with Python 3.

The various trade-offs and potential pitfalls involved in removing the GIL are by now very well known, not just here on HN but among the Python core developers who will ultimately do the heavy lifting when it comes to doing this work.

Re: Python 3.11 vs 3.10 performance

#270

Post by texaslonghorn5 »

int_19h wrote: post #266
ChadNauseam wrote: post #223

Earlier quoted context omitted.

Hold mine :D https://github.com/anchpop/genomics_viz/blob/master/genomics... That's one expression because it used to be part of a giant comprehension, but I moved it into a function for a bit more readability. I'm considering moving it back just for kicks though. My philosophy is: if you're only barely smart enough to code it, you aren't smart enough to debug it. Therefore, you should code at your limit, to force yo…

Yours is nice and readable. Parents' one is not, but it feels like the indentation is deliberately confusing. I'd lay it out like so: tags = list(set([ nel for subli in [ mel for subl in [ [[jel.split('/')[2:] for jel in el] for el in classified ] for mel in subl ] for nel in subli if nel ])) Still not very readable, tho. But that's largely due to Python's outputs-first sequence comprehension syntax being a mess that…

Does the new walrus := in python solve your problem?
Post reply on HN