Live data from Hacker News

Python performance myths and fairy tales

lwn.net

201–210 of 221 posts

Re: Python performance myths and fairy tales

#201
post #16

I think an important bit of context here is that computers are very, very good at speculative happy-path execution. The examples in the article seem gloomy: how could a JIT possibly do all the checks to make sure the arguments aren’t funky before adding them together, in a way that’s meaningfully better than just running the interpreter? But in practice, a JIT can create code that does these checks, and modern proces…

There's an obvious answer - run everything on GPUs. Each speculative branch runs in parallel on its own core, and you add a layer of super-fast branch switching when a branch runs into a problem.

Given the current state of computing, I am unable to state definitively if this suggestion is satire.

Re: Python performance myths and fairy tales

#202
> Python is fast enough for some tasks, he said, which is why there are so many people using it and attending conferences like EuroPython.

The conclusion is logically flawed: it conflates language popularity with performance, and conference attendance and widespread use are sociological indicators, not evidence of Python's performance. Conflating the two is intellectually negligent.

Additionally, Python's speed is largely due to C extensions handling performance-critical tasks, not the interpreter itself. Perl, however, is often faster even in pure code, especially for text processing and regex, thanks to its optimized engine, making it inherently quicker in many common scenarios.

Re: Python performance myths and fairy tales

#203

Earlier quoted context omitted.

I maintain a program written in Python that is faster than the program written in C that it replaces. The C version can do a lot more operations, but it amounts to enumerating 2^N alternatives when you could enumerate N alternatives instead. Certainly my version would be even faster if I implemented it in C, but the gains of going from exponential to linear completely dominate the language difference.

So you're saying two different programs implementing two different algorithms perform differently and that lets you draw a conclusion about how the underlying language/compliers/interpreters behave? Have you ever heard of a controlled variable?

I must have been unclear. C is faster. The Python program would be faster if I reimplemented it in C. However, Python makes it so much easier to transform the problem into a linear form that it’s a bigger win to use Python than to continue maintaining the C version.

And that’s my point: raw execution speed is only helpful when you’re executing the right thing. Don’t discount how much easier it can be to implement the right thing in Python.

Re: Python performance myths and fairy tales

#204
post #198
post #87

Earlier quoted context omitted.

Honestly that seems Sisyphean to me. The market doesn't want a "performant subset". The market is very well served by performant languages. The market wants Python's expressivity. The market wants duck typing and runtime-inspectable type hierarchies and mutable syntax and decorators. It loves it. It's why Python is successful. My feeling is that numba has exactly the right tactic here. Don't try to subset python from…

The market can have that in plenty of dynamic languages with JIT compilers, including Python, if there was more PyPy love from the community.

[deleted]

Re: Python performance myths and fairy tales

#206
post #198
post #87

Earlier quoted context omitted.

Honestly that seems Sisyphean to me. The market doesn't want a "performant subset". The market is very well served by performant languages. The market wants Python's expressivity. The market wants duck typing and runtime-inspectable type hierarchies and mutable syntax and decorators. It loves it. It's why Python is successful. My feeling is that numba has exactly the right tactic here. Don't try to subset python from…

The market can have that in plenty of dynamic languages with JIT compilers, including Python, if there was more PyPy love from the community.

PyPy isn't unloved, just incompatible. The routine experience with PyPy is that you try your toy benchmark, drop your jaw in amazement, then run your whole app and it fails. Usually it's because you have a upstream dependency on a native module without a PyPy port, or with an incompatible implementation (I've personally run into trouble with ctypes code that didn't quite work the same in PyPy).

But there are a non-trivial number of genuine behavioral differences too. A big delta is garbage collection. CPython is refcounted and greedy, you can write a loop doing more-than-just-heap allocation (like, opening files or something), and it works fine because objects get destroyed (closed) as they're no longer used. The same loop in PyPy runs into resource exhaustion because the relevant resource (file descriptors) runs out before the heap does and the GC doesn't run.

The community has tried to address this particular glitch with cryptic admonishments about using "with" that no one quite understands or can explain, but it hasn't taken and it remains true that PyPy basically doesn't work for anything big unless you develop in PyPy. And honestly I don't see a lot of attention to the issue, it's like PyPy doesn't think it has to be 100% compatible to win. Well, it does.

But getting back to my upthread poing: numba doesn't have those properties. Numba is just CPython, with the added feature that when you know you need a fast loop for something you write your python code to look like C and put a @jit decorator on it.

Re: Python performance myths and fairy tales

#207

> Python is fast enough for some tasks, he said, which is why there are so many people using it and attending conferences like EuroPython. The conclusion is logically flawed: it conflates language popularity with performance, and conference attendance and widespread use are sociological indicators, not evidence of Python's performance. Conflating the two is intellectually negligent. Additionally, Python's speed is la…

[dead]

Re: Python performance myths and fairy tales

#208
post #66

I know I am going to get some hate for this from the "Python-stans" but..."python" and "performance" should never be associated with each other, and same for any scripting/interpreted programming language. Especially if it has a global interpreter lock. While performance (however you may mean that) is always a worthy goal, you may need to question your choice of language if you start hitting performance ceilings. As…

Have you read the fine article?

I did, thought it was pretty good an upvoted it.

Re: Python performance myths and fairy tales

#209
post #55
post #25

Earlier quoted context omitted.

Mojo NOT being open-source is a complete non-starter.

More of a question of /will/ Mojo eventually be entirely open source, chunks of it already are. The intent from Modular is eventually it will be, just not everything all at once and not whilst they're internally doing loads of dev for their own commercial entity. Which seems fair enough to me. Importantly they have open sourced lots of the stdlib which is probably what anyone external would contribute to or want to c…

Building on vapor carries a lot of risk. For all anyone knows they take a page from Mathworks and it winds up costing $20k+/year/license if you aren't an academic.

Re: Python performance myths and fairy tales

#210
post #48

Earlier quoted context omitted.

That makes it so that in absolute terms, Python is not as slow as you might naively expect. But we don't measure programming language performance in absolute terms. We measure them in relative terms, generally against C. And while your Python code is speculating about how this Python object will be unboxed, where its methods are, how to unbox its parameters, what methods will be called on those, etc., compiled code i…

I love this “real work”. Real work, like writing linked lists, array bounds checking, all the error handling for opening files, etc, etc? There is a reason Python and C both have a use case, and it’s obvious Python will never be as fast as C doing “1 + 1”. The real “real work” is in getting stuff done, not just making sure the least amount of cpu cycles are used to accomplish some web form generation. Anyway, I think…

"There is a reason Python and C both have a use case [..]"

If you mean historically, then yes, but I don't think there is an inherent reason why we couldn't have a language as convenient as Python and fast as C.

Post reply on HN