Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python 3.11 vs 3.10 performance

github.com

Post reply on HN
Print view View original

341–350 of 460 posts

Re: Python 3.11 vs 3.10 performance

#341

Post by pclmulqdq »

FridgeSeal wrote: post #335
pclmulqdq wrote: post #331

Earlier quoted context omitted.

I'm not the person you responded to, but I think the gist of it is: what it is is not defined by how it is used. Python, at its core, is a scripting language, like awk and bash. The other uses don't change that. Occasionally, a technology breaks out of its intended domain. Python is one of these - it plays host to lots of webservers, and even a filesystem (dropbox). Similarly, HTML is a text markup language, but that…

If we’re going to leave Python as a scripting language (fine by me), can we get the machine learning community to swap to something better suited? It strikes me as a bit of a waste of resources to keep stapling engineering effort into the Python ML/data ecosystem when it’s basically a crippled language capable of either: mindlessly driving C binaries, or scripting simple tasks. What other performance, feature and tec…

From what I can tell, the ML community is moving toward Julia. I don't think anyone predicted that they would end up locked into Python so heavily.

Re: Python 3.11 vs 3.10 performance

#342

Post by kortex »

coliveira wrote: post #163
ianbicking wrote: post #82

Earlier quoted context omitted.

Python is compiled to a VM (that is CPython). The semantics and implementation of the VM haven't prioritized performance as much as some other systems. Here we're seeing improvements in the implementation. But the semantics will be the hard part, as those semantics limit the performance. For instance "a + b" is (I believe) compiled into bytecodes that pretty much follow the expression. But the implementation of "add"…

Dynamic languages like Javascript have solved this problem by essentially caching the resolution of a particular expression that is executed very often. I don't see why this cannot be done in Python.

This is exactly what a lot of the current round of optimization work focuses on. Things like optimistic dispatch on hot loops that fall back to fully dynamic on a miss.

Re: Python 3.11 vs 3.10 performance

#343

Post by dahart »

BeetleB wrote: post #147
atwood22 wrote: post #79

Earlier quoted context omitted.

> Python was meant as a scrypting language that was easy to learn and work with on all levels. Being fast isn't contradictory with this goal. If anything, this is a lesson that so many developers forget. Things should be fast by default.

> Things should be fast by default. In over 90% of my work in the SW industry, being fast(er) was of no benefit to anyone. So no, it should not be fast by default.

> no, it should not be fast by default.

Maybe better to elaborate on what it should be, if not fast? Surely you aren’t advocating things should be intentionally slow by default, or carelessly inefficient?

There’s a valid tradeoff between perf and developer time, and it’s fair to want to prioritize developer time. There’s a valid reason to not care about fast if the process is fast enough that a human doesn’t notice.

That said, depends on what your work is, but defaulting to writing faster, more efficient code might benefit a lot of people indirectly. Lower power is valuable for server code and for electricity bills and at some level for air quality in places where power isn’t renewable. Faster benefits parallel processing, it leaves more room for other processes than yours. Faster means companies and users can buy cheaper hardware.

Re: Python 3.11 vs 3.10 performance

#344

Post by pclmulqdq »

metadat wrote: post #334
pclmulqdq wrote: post #331

Earlier quoted context omitted.

I'm not the person you responded to, but I think the gist of it is: what it is is not defined by how it is used. Python, at its core, is a scripting language, like awk and bash. The other uses don't change that. Occasionally, a technology breaks out of its intended domain. Python is one of these - it plays host to lots of webservers, and even a filesystem (dropbox). Similarly, HTML is a text markup language, but that…

Python has already become a lot more than a scripting language. To say today that scripting is it's core identity seems naive at best. Yes, it has roots but has object oriented and functional facets which do not exist in awk or bash. Pandas, numpy, scipy, tensorflow. All of these go way beyond what is possible with a scripting language. Since when is the runtime performance of a script a serious concern? Why is it a…

Awk is a functional language with a surprising number of features. Bash, not so much. Developers really do care about runtime performance of scripting languages: They often wait for scripts to finish running before doing other work, and if the sum of the time it takes to write and execute a script is too long, they will look for an alternative.

All of the libraries you have cited are from the ML and stats communities, and they are not core language features. From what I understand, ML folks like Python because it is fast to play with and get results. In other words, they like it because it is a scripting language.

Personally, I like that Python has kept the GIL so far because I would never run a 24/7 server in Python and I am happy to use it very frequently for single-threaded scripting tasks.

Edit: I didn't decide that Python was a scripting language. The Python maintainers did. The point is that the identity of a project doesn't flow down from its use cases.

Edit 2: I should have said "the identity of a project doesn't flow down from its users."

Re: Python 3.11 vs 3.10 performance

#345

Post by galangalalgol »

jolux wrote: post #336
galangalalgol wrote: post #329

Earlier quoted context omitted.

I like Julia but its easy to write slow julia unless you keep the performance tips in mind. Arrays are horribly slow, tuples are much faster, but having tuples be a multiple of 128 bytes adds 10% or more to speed. I honestly don't understand how much slower arrays are. Its like 30x or similar.

That’s bizarre, are they implemented as linked lists or something? Why would arrays be that slow?

They are not guaranteed to be contiguous, but they supposedly are if you specify a primative type and initialize them all at once. Tuples have a constant size, are contiguous, and immutable. So lots of optimization opportunity.

Re: Python 3.11 vs 3.10 performance

#346

Post by bulatb »

bornfreddy wrote: post #220
hn_saver wrote: post #103

Earlier quoted context omitted.

Threading IS concurrency. When you say "real" concurrency, you actually mean parallelism.

Not in CPython it isn't. Threading in CPython doesn't allow 2 threads to run concurrently (because of GIL). As GP correctly stated, you need multiprocessing (in CPython) for concurrency.

They're emphasizing a precise distinction between "concurrent" (the way it's structured) and "parallel" (the way it runs).

Concurrent programs have multiple right answers for "Which line of computation can make progress?" Sequential execution picks one step from one of them and runs it, then another, and so on, until everything is done. Whichever step is chosen from whichever computation, it's one step per moment in time; concurrency is only the ability to choose. Parallel execution of concurrent code picks steps from two or more computations and runs them at once.

Because of the GIL, Python on CPython has concurrency but limited parallelism.

Re: Python 3.11 vs 3.10 performance

#347

Post by BeetleB »

dahart wrote: post #343
BeetleB wrote: post #147

Earlier quoted context omitted.

> Things should be fast by default. In over 90% of my work in the SW industry, being fast(er) was of no benefit to anyone. So no, it should not be fast by default.

> no, it should not be fast by default. Maybe better to elaborate on what it should be, if not fast? Surely you aren’t advocating things should be intentionally slow by default, or carelessly inefficient? There’s a valid tradeoff between perf and developer time, and it’s fair to want to prioritize developer time. There’s a valid reason to not care about fast if the process is fast enough that a human doesn’t notice.…

> Maybe better to elaborate on what it should be, if not fast?

It should satisfy the needs of the customer and it should be reasonably secure.

Everything else is a luxury.

My point was that in most of my time in the industry being faster would not have benefited my customers in any manner worth measuring.

I'm not anti-performance. I fantasize about how to make my software faster to maintain my geek credentials. But neither my bosses nor my customers pay for it.

If a customer says they want it faster we'll oblige.

Re: Python 3.11 vs 3.10 performance

#349

Post by Galanwe »

tempest_ wrote: post #306
tyingq wrote: post #250

Earlier quoted context omitted.

Getting rid of the GIL will also immediately expose all the not-thread-safe stuff that currently exists, so there's a couple of waves you would need before it would be broadly usable.

Cool, they should start now. As a python dev, pythons multiprocess/multithreading story is one the largest pain points in the language. Single threaded performance is not that useful while processors have been growing sideways for 10 years. I often look at elixir with jealousy.

> As a python dev, pythons multiprocess/multithreading story is one the largest pain points in the language.

Hmm, how is that so?

As a python dev as well, I don't have much complaint with multiprocessing.

The API is simple, it works OK, the overall paradigm is simple to grok, you can share transparently with pickle, etc.

Re: Python 3.11 vs 3.10 performance

#350

Post by Supermancho »

pclmulqdq wrote: post #344
metadat wrote: post #334

Earlier quoted context omitted.

Python has already become a lot more than a scripting language. To say today that scripting is it's core identity seems naive at best. Yes, it has roots but has object oriented and functional facets which do not exist in awk or bash. Pandas, numpy, scipy, tensorflow. All of these go way beyond what is possible with a scripting language. Since when is the runtime performance of a script a serious concern? Why is it a…

Awk is a functional language with a surprising number of features. Bash, not so much. Developers really do care about runtime performance of scripting languages: They often wait for scripts to finish running before doing other work, and if the sum of the time it takes to write and execute a script is too long, they will look for an alternative. All of the libraries you have cited are from the ML and stats communities…

>> To say today that scripting is it's core identity seems naive at best. Yes, it has roots but has object oriented and functional facets which do not exist in awk or bash. Pandas, numpy, scipy, tensorflow. All of these go way beyond what is possible with a scripting language.

No, it is not. It's what you can do with a scripting language, which is a fairly straightforward functional categorization. If you are very familiar with it and really want to just ignore the performance issues and don't mind strange workarounds (eg elaborate libraries like Twisted), you probably use python.

> The point is that the identity of a project doesn't flow down from its use cases.

Use cases are what determines the identity of a language. It takes a massive and ongoing marketing campaign to convince people otherwise, with limited success without use-cases. Python is popular because it's a scripting language with a relatively low learning curve (one true way to do things philosophy). That's it. It will improve over time, but it's been slow going...and that's in comparison to the new Java features!

Haskell is a shining example of how the language identity is foisted upon the public until you are convinced enough to try it. It doesn't take long to learn it's a nightmare of incomplete features and overloaded idioms for common tasks, so it isn't used. There aren't good use-cases for a language with haskell's problems, so the developer community and industry-at-large avoids it.

Post reply on HN