Live data from Hacker News

Python 3.11 vs 3.10 performance

github.com

401–410 of 460 posts

Re: Python 3.11 vs 3.10 performance

#401

Earlier quoted context omitted.

O, I agree. As I said, when python and its uses scaled, it became quite necessary to make it fast. I like that it will be fast as well and I am not happy that it is slow at the moment. My point is that there are reasons why it was not optimized in the beginning and why this process of optimizations has started now.

Back when Python was started there was really C or C++ for optimized programs and scripting languages like Python and Perl. But since Python had the ability for C Extensions it allowed it to bypass those problems. Since Python was easy to learn both web developers and scientists to learn. Then financial organizations started to get interested and that’s really how Python cemented itself. What exactly do you do with P…

Scheme and Lisp already existed.

Re: Python 3.11 vs 3.10 performance

#402
post #358

Earlier quoted context omitted.

Well, good generics and sane error handling, for two examples.

Go recently got pretty decent generics. I'm with you on the error handling though. At least it's explicit. Go also has a plethora of really useful libraries, and that along with good editor and build tooling is probably the real draw of the language for me.

Version 1.0 generics, still better than nothing, though.

Re: Python 3.11 vs 3.10 performance

#403

Earlier quoted context omitted.

Yes, you can spend a large amount of time making things faster. But note that Go's GC is fast, even though it is simple. It's not the fastest, but it is acceptably fast.

Funny you should pick that example in a sub thread that you started with an assertion that code should be fast from by default. Go’s GC was intentionally slow at first. They wanted to get it right THEN make it fast. No offense but you’re not making a strong case. You’re sounding like an inexperienced coder that hasn’t yet learned that premature optimization is bad.

Far from it, Go was designed to be optimizable form the start. The GC was obviously not optimal, but the language semantics were such that the GC can be replaced with a better one with relatively minimal disruption.

Of course one can't release optimal code from version one, that would be absurd.

Also your last sentence is extremely condescending.

Re: Python 3.11 vs 3.10 performance

#404
post #336

Earlier quoted context omitted.

I like Julia but its easy to write slow julia unless you keep the performance tips in mind. Arrays are horribly slow, tuples are much faster, but having tuples be a multiple of 128 bytes adds 10% or more to speed. I honestly don't understand how much slower arrays are. Its like 30x or similar.

That’s bizarre, are they implemented as linked lists or something? Why would arrays be that slow?

They aren't, which is why the statement doesn't make sense haha. The difference is really just that tuples of isbits types can be stack allocated (and tuples of isbits types are isbits, so they can nest, etc.), and so in some cases with sufficiently small amounts of data, creating stack allocated objects is much faster than creating heap allocated objects.

But if you compare concretely typed heap allocated arrays in Julia and C, there's no real performance difference (Steven Johnson has a nice notebook displaying this https://scpo-compecon.github.io/CoursePack/Html/languages-be..., and if you see the work on LoopVectorization.jl it makes it really clear that the only major difference is that C++ tends to know how to prove non-aliasing (ivdep) in a bit more places (right now)). So the real question is, did you actually want to use a heap allocated object? I think this really catches people new to performance optimization off guard since in other dynamic languages you don't have such control to "know" things will be placed on the stack, so you generally heap allocate everything (Python even heap allocates numbers), which is one of the main reasons for the performance difference against C. Julia gives you the tools to write code that doesn't heap allocate objects, but also makes it easy to heap allocate objects (because if you had to `malloc` and `free` everywhere... well you'd definitely lose the "like Python" and and be much closer to C++ or Rust in terms of "ease of use", which would defeat the purpose for many cases). But if you come from a higher level language, there's this magical bizarre land of "things that don't allocate" (on the heap) and so you learn "oh I got 30x faster from Python, but then someone on a forum showed me how to do 100x better by not making arrays?", which is somewhat obvious from a C++ mindset but less obvious from a higher level language mindset.

And FWIW, this is probably the biggest performance issue newcomers run into, and I think one of the things to which a solution is required to make it mainstream. Thankfully, there's already prototype PRs that are well underway, for example https://github.com/JuliaLang/julia/pull/43573 automatically stack-allocates small arrays which can prove certain escape properties, https://github.com/JuliaLang/julia/pull/43777 is looking to hoist allocations out of loops so even if they are required they are minimized in the loop contexts automatically, etc. The biggest impediment is that EscapeAnalysis.jl is not in Base, and it requires JET.jl which is not in Base, and so both of those need to be made "Base friendly" to join the standard library and then the compiler can start to rely on its analysis (which will be nice because JET.jl can do things like throw more statically-deducible compile time errors, another thing people ask for with Julia). There's a few people who are working really hard on doing this, so it is a current issue but it's a known one with prototyped solutions and a direction to get it into the shipped compiler. When that's all said and done, of course "good programming" will still help the compiler in some cases, but in most people shouldn't have to worry about stack vs heap (it's purposefully not part of the user API in Julia and considered a compiler detail for exactly this reason, so it's non-breaking for the compiler to change where objects live and improve performance over time).

Re: Python 3.11 vs 3.10 performance

#405

Earlier quoted context omitted.

That’s what I meant, if JavaScript hadn’t been trapped in the browser and allowed to call C, maybe there wouldn’t have been so much investment in making it fast. But still, it’s kind of surprising that PHP has a JIT and Python doesn’t (official implementation I mean, not PyPy).

Python has a lot more third-party packages that are written in native code. PHP has few partly because it just isn't used much outside of web dev, and partly because the native bits that might be needed for web devs, like database libraries, are included in the core distro.

Because it has to, there is no other way to make it fast, as typical two language dilemma.

Re: Python 3.11 vs 3.10 performance

#406

Earlier quoted context omitted.

> I don't see how "asyncio is annoying and can only be used for a fraction of scenarios everywhere else too, not just here" is anything other than reinforcement of what I said. Well, I didn't try to refute what you wrote (for one, it's clearly a personal, subjective opinion). I asked what I've asked merely to clarify whether your issue is with Python's asyncio (e.g. Python got it wrong) or with the tradeoffs inherent…

> I've asked merely to clarify whether your issue is with Python's asyncio (e.g. Python got it wrong) or with the tradeoffs inherent in async io APIs in general (regardless of Python) Both, but the latter part is contextual. > I, for one, am fine with async APIs in JS Correct me if you think I'm wrong, but JS in its native environment (the browser) never had access to the OS thread and process scheduler, so the conce…

>1. Python has never had that constraint

Which is neither here, nor there. Python had another big constraint, the GIL. So threads there couldn't go so far as async would. But even environments with threads (C#, Rust) also got big into async in the same style.

>2. Python's asyncio in particular is a shitty hammer that only works on special asyncio-branded nails

Well, that's also the case with C#, JS, and others with similar async style (aka "colored functions"). And that's not exactly a problem, as much as a design constraint.

Re: Python 3.11 vs 3.10 performance

#407
post #343

Earlier quoted context omitted.

> no, it should not be fast by default. Maybe better to elaborate on what it should be, if not fast? Surely you aren’t advocating things should be intentionally slow by default, or carelessly inefficient? There’s a valid tradeoff between perf and developer time, and it’s fair to want to prioritize developer time. There’s a valid reason to not care about fast if the process is fast enough that a human doesn’t notice.…

> Maybe better to elaborate on what it should be, if not fast? It should satisfy the needs of the customer and it should be reasonably secure. Everything else is a luxury. My point was that in most of my time in the industry being faster would not have benefited my customers in any manner worth measuring. I'm not anti-performance. I fantasize about how to make my software faster to maintain my geek credentials. But n…

Clean code, easy to maintain is not luxury when talking about a programming language implementation.

Re: Python 3.11 vs 3.10 performance

#408
post #100

Earlier quoted context omitted.

> 25% faster is still 25% faster, even if the code would have been 100x faster to begin with in another language.) And that's even assuming that the code would have existed at all in another language. The thing about interpreted GC languages is that the iterative loop of creation is much more agile, easier to start with and easier to prototype in than a compiled, strictly typed language.

I think this is the best explanation for its popularity. Sure, it runs slow, but it's very fast to write. The latter is usually more important in the world of business, for better or worse.

Not just in the world of business - a lot of developers play with these things in their spare time before promoting them to production tech; if it takes ages to write a silly mp3 player or some other hobby project, they'll use something else instead.

Re: Python 3.11 vs 3.10 performance

#409

Earlier quoted context omitted.

Awk is a functional language with a surprising number of features. Bash, not so much. Developers really do care about runtime performance of scripting languages: They often wait for scripts to finish running before doing other work, and if the sum of the time it takes to write and execute a script is too long, they will look for an alternative. All of the libraries you have cited are from the ML and stats communities…

"Personally, I like that Python has kept the GIL so far because I would never run a 24/7 server in Python and I am happy to use it very frequently for single-threaded scripting tasks." Just as a side-note - my prior gig used Python on both the Server and Data Collection industrial systems. It was very much a 24x7x365 must-never-go-down type of industrial application, and, particularly when we had a lot of data-source…

I know that people do use python for 24/7 "must not fail" applications. I'm just not smart enough to write python that I would trust like that. Python comes with a tremendous number of foot guns and you have to find them because there is no compiler to help, and it can be a real pain to try to understand what is happening in large (>10,000 line) python programs.
Post reply on HN