Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python 3.11 vs 3.10 performance

github.com

Post reply on HN
Print view View original

431–440 of 460 posts

Re: Python 3.11 vs 3.10 performance

#431

Post by jfim »

jart wrote: post #225
antman wrote: post #208

There was always a denial of removing the Global Interpreter Lock because it would decrease single threaded Python speed for which most people din’t care. So I remember a guy recently came up with a patch that both removed GIL and also to make it easier for the core team to accept it he added also an equivalent number of optimizations. I hope this release was is not we got the optimizations but ignored the GIL part.…

I use a beautiful hack in the Cosmopolitan Libc codebase (x86 only) where we rewrite NOPs into function calls at runtime for all locking operations as soon as clone() is called. https://github.com/jart/cosmopolitan/blob/5df3e4e7a898d223ce... The big ugly macro that makes it work is here https://github.com/jart/cosmopolitan/blob/master/libc/intrin... An example of how it's used is here. https://github.com/jart/cosmopo…

That's a pretty clever hack, nicely done!

Re: Python 3.11 vs 3.10 performance

#432

Post by ModernMech »

josefx wrote: post #372
int_19h wrote: post #261

Earlier quoted context omitted.

Removing GIL also breaks existing native packages, and would require wholesale migration across the entire ecosystem, on a scale not dissimilar to what we've seen with Python 3.

> on a scale not dissimilar to what we've seen with Python 3. If only people had asked for it before the Python 3 migration so it could have been done with all the other breaking and performance harming changes. But no, people only started to ask for it literally yesterday so it just could not ever be done, my bad. /sarcasm If anything the whole Python 3 migration makes any argument against the removal of the GIL app…

I think probably the reason is that in 2008, consumer chips were still mostly single or dual core. Today we have consumer grade chips that have dozens of cores, so the calculus has changed.

Re: Python 3.11 vs 3.10 performance

#433

Post by smcl »

michaelcampbell wrote: post #428
smcl wrote: post #412

Earlier quoted context omitted.

That's not actually a bad observation even though you were downvoted. I think in the end it's technically ambiguous but most people would be able to see what is meant especially given the previous performance is present (i.e. for benchmark "deltablue" we have columns "12.4 ms" and "6.35 ms (1.96x faster)". But you're right, I think "1.96x as fast" sounds more correct. edit: but now that I think about it "X as fast" o…

It's an OCD pet peeve of mine. Worse yet is "2x slower!" To me, 1x slower is "stopped". I know they MEAN "1/2 the speed", but it still bugs me.

Oh that's like the US thing of saying something as "10 times less than ..." rather than "one tenth of ..."

Re: Python 3.11 vs 3.10 performance

#434

Post by smcl »

winstonewert wrote: post #427
smcl wrote: post #387

Earlier quoted context omitted.

Right but if they post just that part, they're probably heavily implying that now is not the time to optimize. I've seen way more people using it to argue that you shouldn't be focussing on performance at this time, than saying "sometimes you gotta focus on that 3% mentioned in the part of the quote that I deliberately omitted"

No, they don't deliberately omit the part of the quote. They are either unaware of that part of the quote or don't think it matters to the point they are making. Yes, if you quote Knuth here (whether the short quote or a longer version) you are probably responding to someone whom you believe is engaged in premature optimization. It remains that the person quoting Knuth isn't claiming that there isn't such a thing as…

I dunno, guess it's hard to say given we're talking about our own subjective experiences. I completely believe there are people who just know the "premature optimization is the root of all evil" part and love to use it because quoting Knuth makes them sound smart. And I'm sure there are also people know it all and who quote that part in isolation (and in good faith) because they want to emphasise that they believe you're jumping the gun on optimization.

But either way I think the original statement is so uncontroversial and common-sense I actually think it doesn't help any argument unless you're talking to an absolutely clueless dunce or unless you're dealing with someone who somehow believes every optimization is premature.

Re: Python 3.11 vs 3.10 performance

#435

Post by maxbond »

lmm wrote: post #312
maxbond wrote: post #109

Earlier quoted context omitted.

This certainly does not mean, "tolerate absurd levels of technical debt, and only ever think about performance in retrospect."

> tolerate absurd levels of technical debt In my experience it's far more common for "optimizations" to be technical debt than the absence of them. > only ever think about performance in retrospect From the extra context it pretty much does mean that. "but only after that code has been identified" - 99.999% of programmers who think they can identify performance bottlenecks other than in retrospect are wrong, IME.

Well it's entirely possible that Knuth and I disagree here, but if you architect an application without thinking about performance, you're likely going to make regrettable decisions that you won't be able to reverse.

It is not possible to predict bottlenecks in computation, no. But the implications of putting global state behind a mutex in a concurrent application should be clear to the programmer, and they should think seriously before making a choice like that. If you think of a different way to do that while the code is still being written, you'll avoid trapping yourself in an irreversible decision.

Re: Python 3.11 vs 3.10 performance

#436

Post by p10_user »

pjmlp wrote: post #394
oofbey wrote: post #330

Earlier quoted context omitted.

On the flip side, if your workload can be parallelized across thousands of cores, python has about the best CUDA support anywhere.

That would be C++ and Fortran actually.

But the python bindings are great and useful to many, so Python gets to be added to the list.

Re: Python 3.11 vs 3.10 performance

#437

Post by p10_user »

metadat wrote: post #334
pclmulqdq wrote: post #331

Earlier quoted context omitted.

I'm not the person you responded to, but I think the gist of it is: what it is is not defined by how it is used. Python, at its core, is a scripting language, like awk and bash. The other uses don't change that. Occasionally, a technology breaks out of its intended domain. Python is one of these - it plays host to lots of webservers, and even a filesystem (dropbox). Similarly, HTML is a text markup language, but that…

Python has already become a lot more than a scripting language. To say today that scripting is it's core identity seems naive at best. Yes, it has roots but has object oriented and functional facets which do not exist in awk or bash. Pandas, numpy, scipy, tensorflow. All of these go way beyond what is possible with a scripting language. Since when is the runtime performance of a script a serious concern? Why is it a…

But all the “magic” that makes the scientific stack so great is largely due to numpy (and many other Fortran or c driven code) being fantastic. Python is the glue scripting language for organizing and calling procedures. That’s,IMO, it’s original and core purpose.

Now the fact that you can do some fun metaprogramming shenanigans in Python just speaks to how nice it is to write.

Re: Python 3.11 vs 3.10 performance

#438

Post by HdS84 »

jmcgough wrote: post #277
CobrastanJorji wrote: post #228

Earlier quoted context omitted.

I've been learning Rust recently. There are a number of things about the language that I dislike, but its error messages are an absolute joy. They clearly put an incredible amount of effort into them, and it really shows.

I wrote 2-3 of those and I really wish more languages had a similar approach with their standard library. When you get an error there's a verbose explanation you can ask for, which describes the problem, gives example code and suggests how you can fix it. The language has a longer ramp-up period because it contains new paradigms, so little touches like this help a lot in onboarding new devs.

I am a hug fan of ravendb's founder ayende. He always stresses how useful good error messages are, especially for self service. And he is certainly on spot with that, e.g "could not execute Mode None" is much less helpful than " environment variable X not set, could not determine Mode. Please set this as MODE=Agressive or MODE=Passive".

Re: Python 3.11 vs 3.10 performance

#439

Post by disgruntledphd2 »

Mehdi2277 wrote: post #355
pclmulqdq wrote: post #341

Earlier quoted context omitted.

From what I can tell, the ML community is moving toward Julia. I don't think anyone predicted that they would end up locked into Python so heavily.

That's not been my experience much at all work or research wise. Tensorflow, pytorch, jax are still very dominant. I've worked at several companies and interviewed at several dozen for ML roles. They have 100% been python/c++ for ml. I'd be impressed if even 2% of ML engineers used Julia.

I feel like Julia will take more of the R people than the Python people, to be honest.

Re: Python 3.11 vs 3.10 performance

#440

Post by michaelcampbell »

smcl wrote: post #433
michaelcampbell wrote: post #428

Earlier quoted context omitted.

It's an OCD pet peeve of mine. Worse yet is "2x slower!" To me, 1x slower is "stopped". I know they MEAN "1/2 the speed", but it still bugs me.

Oh that's like the US thing of saying something as "10 times less than ..." rather than "one tenth of ..."

Exactly that. I'm in the US and see it, but didn't know it was a US "thing". But yes, that.
Post reply on HN