Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is 1.3x faster by just adjusting some compiling options for libpython

bugs.python.org

Post reply on HN
Print view View original

51–60 of 105 posts

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#51

Post by nwmcsween »

dheera wrote: post #6
Scaevolus wrote: post #4

This is true for _libpython_ (the shared library version), which is the default on some distros (RedHat, Fedora, Arch), but many others (Debian, Ubuntu) use statically linked Python and never paid this performance tax.

I think using Pypy instead of CPython will give you several times the performance boost as any of this.

Assuming pypy does what it says it does: Python -> RPython -> C -> LLVM Clang -> LLVM IR -> 'JIT', you're still paying a large tax compared to something like RPython -> Some IR -> 'JIT'

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#52

Post by cutler »

stereo wrote: post #3

Text if you don't want to visit Facebook: Summary: Python is 1.3x faster when compiled in a way that re-examines shitty technical decisions from the 1990s. ELF is the executable and shared library format on Linux and other Unixy systems. It comes to us from 1992's Solaris 2.0, from back before even the first season of the X-Files aired. ELF files (like X-Files) are full of barely-understood horrors described only in…

Hitting Reader quickly on load in Firefox also gets you the text.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#53

Post by Dylan16807 »

geofft wrote: post #7

Related: https://developers.redhat.com/blog/2020/06/25/red-hat-enterp... > This article focuses on one specific performance improvement in the python38 package. As we'll explain, Python 3.8 is built with the GNU Compiler Collection (GCC)'s -fno-semantic-interposition flag. Enabling this flag disables semantic interposition, which can increase run speed by as much as 30%. (not logged in to FB, so maybe TFA is a refere…

Also https://pythonspeed.com/articles/faster-python/

And having just read that yesterday, I was very confused to open this post and immediately see complaints about the author's tone.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#54

Post by coldtea »

jchw wrote: post #11

There’s nothing wrong with this post factually, but the tone sucks. It has an immensely combative energy for what is not really a charged subject matter. Like sure. Today, a lot of the historical reasons for things seem silly and irrelevant. At one point, they did not seem silly and irrelevant. For compatibility with stuff sticking around from those days, we get some performance penalties that are not strictly necess…

>There’s nothing wrong with this post factually, but the tone sucks. It has an immensely combative energy for what is not really a charged subject matter.

Perhaps because the subject matter doesn't matter.

The really important takeaway message is more about the industry/community not paying attention to shitty defaults and winging it for decades, than about the potential speedup and/or this particular mechanism.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#55

Post by coldtea »

rtpg wrote: post #46
CalChris wrote: post #27

Earlier quoted context omitted.

The OP was writing about a 29 year old design decision, and he wasn’t writing about a person. Design decisions don’t have feelings. I found his no holds barred clarity about something as obscure as dynamic linking namespaces made for an easier if still not easy read. But that said, I don’t think dynamic linking is in the ELF spec. I believe that’s a de facto OS + dev tools thing rather than an ELF spec de jure thing.…

This is recalling the old Linus debates, but the aggressiveness _doesn't improve the clarity_, and is basically upping the word count. I'm not tone policing but contesting the premise that "aggressive tone" = "direct". For example >(Windows took a different approach and got it right. In Windows, it's okay for multiple DLLs to provide the same symbol, and there's no sad and desperate effort to pretend that a single na…

>This is recalling the old Linus debates, but the aggressiveness _doesn't improve the clarity_, and is basically upping the word count.

It drives the point home though, and keeps the energy levels higher.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#56

Post by cutler »

mastrsushi wrote: post #48

Python is like the cockroach equivalent of those shell scripting languages that came out of the late 80s to early 90s. Perl, Ruby, PHP, TCL, and Lua have definetly declined over the years. Python's biggest asset seems to be featured rich libraries, rather than the language itself.

Ruby is a much better designed language than Python but the community unfortunately made the mistake of focusing too much on Rails and web development at the expense of diversification. Python is the Skoda of scripting languages.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#57

Post by coldtea »

jordigh wrote: post #44
CalChris wrote: post #37

Earlier quoted context omitted.

I don’t think it’s possible to be an asshole to an inanimate object.

Someone wrote that inanimate object. Someone likes that inanimate object. Someone thinks that inanimate object has reasons to be the way it is. Attacking that inanimate object is not without emotional repercussions to those related to that object.

>Someone wrote that inanimate object. Someone likes that inanimate object.

You'll find somebody liking everything you can name. Even every attrocity has been done by somebody.

Not attacking the person is of course the correct thing.

But if you can't even attack the attrocity or bad choice, we've gone too far with this sensitivity thing.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#58

Post by dbt00 »

The people who did the work on finding and implementing this speedup talk about it here.

https://fedoraproject.org/wiki/Changes/PythonNoSemanticInter...

They then proposed it for upstreaming here: https://bugs.python.org/issue38980

And that's the new default.

The link is to a facebook post by someone who neither discovered nor implemented this change.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#59

Post by coldtea »

dheera wrote: post #6
Scaevolus wrote: post #4

This is true for _libpython_ (the shared library version), which is the default on some distros (RedHat, Fedora, Arch), but many others (Debian, Ubuntu) use statically linked Python and never paid this performance tax.

I think using Pypy instead of CPython will give you several times the performance boost as any of this.

Now you have 10 problems.

A reference to the classic line, oft attributed to JWZ:

"Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems".

The point being, switching from CPython to PyPy does not simply give you the performance boost and that's it. It comes with its own tradeoffs, including:

(a) trading speed for much more memory consumption

(b) slower C FFI - important for all kinds of Python workflows (e.g. Pandas, Numpy, and so on)

(c) behind mainline CPython releases, and with subtle incompatibilities

(d) slower startup times (due to the JITting involved)

(e) different garbage collection model (and less predictable)

(f) less support (from companies, distros, etc), fewer ports, less manpower to port quickly to new platforms (e.g. Apple's M1).

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#60

Post by chrisseaton »

coldtea wrote: post #55
rtpg wrote: post #46

Earlier quoted context omitted.

This is recalling the old Linus debates, but the aggressiveness _doesn't improve the clarity_, and is basically upping the word count. I'm not tone policing but contesting the premise that "aggressive tone" = "direct". For example >(Windows took a different approach and got it right. In Windows, it's okay for multiple DLLs to provide the same symbol, and there's no sad and desperate effort to pretend that a single na…

> This is recalling the old Linus debates, but the aggressiveness _doesn't improve the clarity_, and is basically upping the word count. It drives the point home though, and keeps the energy levels higher.

> It drives the point home though

Is the argument too weak to do that on its own without the abuse?

Post reply on HN