Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python performance myths and fairy tales

lwn.net

Post reply on HN
Print view View original

121–130 of 221 posts

Re: Python performance myths and fairy tales

#121

Post by Philpax »

ajross wrote: post #101
zozbot234 wrote: post #94

Earlier quoted context omitted.

> The market wants duck typing and runtime-inspectable type hierarchies and mutable syntax and decorators. It loves it. These features have one thing in common: they're only useful for prototype-quality throwaway code, if at all. Once your needs shift to an increased focus on production use and maintainability, they become serious warts. It's not just about performance (though it's obviously a factor too), there's re…

> These features have one thing in common: they're only useful for prototype-quality throwaway code, if at all. As a matter of practice: the python community disagrees strongly. And the python community ate the world. It's fine to have an opinion, but you're not going to change python.

The existence of several type-checkers and Astral's largely-successful efforts to build tooling that pulls Python out of its muck seems to suggest otherwise.

Better things are possible, and I'm hoping that higher average quality of Python code is one of those things.

Re: Python performance myths and fairy tales

#122

Post by zahlman »

mrkeen wrote: post #106
postexitus wrote: post #26

Earlier quoted context omitted.

A more careful reading of the article is required. The first myth is "Python is not slow" - it is debunked, it is slow. The second myth is ""it's just a glue language / you just need to rewrite the hot parts in C/C++" - it is debunked, just rewriting stuff in C/Rust does not help. The third myth is " Python is slow because it is interpreted" - it is debunked, it is not slow only because it is interpreted.

Thanks! As a Python outsider, I was primed for a Python insider to be trying to change my views, not confirm them, and I did indeed misread.

My impression is that GvR conceded a long time ago that Python is slow, and doesn't particularly care (and considers it trolling to keep bringing it up). The point is that in the real world this doesn't matter a lot of the time, at least as long as you aren't making big-O mistakes — and easier-to-use languages make it easier to avoid those mistakes.

For that matter, I recently saw a talk in the Python world that was about convincing people to let their computer do more work locally in general, because computers really are just that fast now.

Re: Python performance myths and fairy tales

#123

Post by didip »

dgan wrote: post #32

"Rewrite the hot path in C/C++" is also a landmine because how inefficient the boundary crossing is. so you really need "dispatch as much as possible at once" instead of continuously calling the native code

These days it's "rewrite in Rust".

Typically Python is just the entry and exit point (with a little bit of massaging), right?

And then the overwhelming majority of the business logic is done in Rust/C++/Fortran, no?

Re: Python performance myths and fairy tales

#124

Post by btown »

Demiurge wrote: post #112
jerf wrote: post #48

Earlier quoted context omitted.

That makes it so that in absolute terms, Python is not as slow as you might naively expect. But we don't measure programming language performance in absolute terms. We measure them in relative terms, generally against C. And while your Python code is speculating about how this Python object will be unboxed, where its methods are, how to unbox its parameters, what methods will be called on those, etc., compiled code i…

I love this “real work”. Real work, like writing linked lists, array bounds checking, all the error handling for opening files, etc, etc? There is a reason Python and C both have a use case, and it’s obvious Python will never be as fast as C doing “1 + 1”. The real “real work” is in getting stuff done, not just making sure the least amount of cpu cycles are used to accomplish some web form generation. Anyway, I think…

To put it another way, I choose Python because of its semantics around dynamic operator definition, duck typing etc.

Just because I don’t write the bounds-checking and type-checking and dynamic-dispatch and error-handling code myself, doesn’t make it any less a conscious decision I made by choosing Python. It’s all “real work.”

Re: Python performance myths and fairy tales

#125

Post by zahlman »

hansvm wrote: post #79

In the "dynamic" section, it's much worse than the author outlines. You can't even assume that the constant named "10" will point to a value which behaves like you expect the number 10 to behave.

I guess you mean "N". 10 is a literal, not a name. The part "N cannot be assumed to be ten, because that could be changed elsewhere in the code" implies well enough that the change could be to a non-integer value. (For that matter, writing `N: int = 10` does nothing to fix that.)

Re: Python performance myths and fairy tales

#126

Post by adsharma »

ajross wrote: post #101
zozbot234 wrote: post #94

Earlier quoted context omitted.

> The market wants duck typing and runtime-inspectable type hierarchies and mutable syntax and decorators. It loves it. These features have one thing in common: they're only useful for prototype-quality throwaway code, if at all. Once your needs shift to an increased focus on production use and maintainability, they become serious warts. It's not just about performance (though it's obviously a factor too), there's re…

> These features have one thing in common: they're only useful for prototype-quality throwaway code, if at all. As a matter of practice: the python community disagrees strongly. And the python community ate the world. It's fine to have an opinion, but you're not going to change python.

That assumes python is one monolithic thing and everyone agrees what it is.

True, the view you express here has strong support in the community and possibly in the steering committee.

But there are differing ideas on what python is and why it's successful.

Re: Python performance myths and fairy tales

#128

Post by kragen »

Twirrim wrote: post #69
aragilar wrote: post #42

Earlier quoted context omitted.

Isn't this just a specific example of the general rule of pulling out repeated use of the same operation in a loop? I'm not sure calls out to C are specifically slow in CPython (given many operations are really just calling C underneath).

The serialisation cost of translating data representations between python and C (or whatever compiled language you're using) is notable. Instead of having the compiled code sit in the centre of a hot loop, it's significantly better to have the loop in the compiled code and call it once https://pythonspeed.com/articles/python-extension-performanc...

You don't have to serialize data or translate data representations between CPython and C. That article is wrong. What's slow in their example is storing data (such as integers) the way CPython likes to store it, not translating that form to a form easily manipulated in C, such as a native integer in a register. That's just a single MOV instruction, once you get past all the type checking and reference counting.

You can avoid that problem to some extent by implementing your own data container as part of your C extension (the article's solution #1); frobbing that from a Python loop can still be significantly faster than allocating and deallocating boxed integers all the time, with dynamic dispatch and reference counting. But, yes, to really get reasonable performance you want to not be running bytecodes in the Python interpreter loop at all (the article's solution #2).

But that's not because of serialization or other kinds of data format translation.

Re: Python performance myths and fairy tales

#129

Post by gen220 »

ic_fly2 wrote: post #28

It’s a good article on speed. But honestly the thing that makes any of my programs slow is network calls. And there a nice async setup goes a long way. And then k8 for the scaling.

I think articles like this cast too wide a net when they say "performance" or " is fast/slow".

A bunch of SREs discussing which languages/servers/runtimes are fast/slow/efficient in comparable production setups would give more practical guidance.

If you're building an http daemon in a traditional three-tiered app (like a large % of people on HN), IME, Python has quietly become a great language in that space, compared to its peers, over the last 8 years.

Re: Python performance myths and fairy tales

#130

Post by adsharma »

ajross wrote: post #87
adsharma wrote: post #73

The most interesting part of this article is the link to SPy. Attempts to find a subset of python that could be made performant.

Honestly that seems Sisyphean to me. The market doesn't want a "performant subset". The market is very well served by performant languages. The market wants Python's expressivity. The market wants duck typing and runtime-inspectable type hierarchies and mutable syntax and decorators. It loves it. It's why Python is successful. My feeling is that numba has exactly the right tactic here. Don't try to subset python from…

My cursory reading is that SPy is generous in what it accepts.

The subset I've been working with is even narrower. Given my stance on pattern matching, it may not even be a subset.

https://github.com/py2many/py2many/blob/main/doc/langspec.md

Post reply on HN