Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is 1.3x faster by just adjusting some compiling options for libpython

bugs.python.org

Post reply on HN
Print view View original

61–70 of 105 posts

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#61

Post by chrisseaton »

coldtea wrote: post #57
jordigh wrote: post #44

Earlier quoted context omitted.

Someone wrote that inanimate object. Someone likes that inanimate object. Someone thinks that inanimate object has reasons to be the way it is. Attacking that inanimate object is not without emotional repercussions to those related to that object.

> Someone wrote that inanimate object. Someone likes that inanimate object. You'll find somebody liking everything you can name. Even every attrocity has been done by somebody. Not attacking the person is of course the correct thing. But if you can't even attack the attrocity or bad choice, we've gone too far with this sensitivity thing.

Why can't they just present the argument without any abuse at all? Why is there ever any need for the abuse?

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#62

Post by phibz »

ineedasername wrote: post #28
jchw wrote: post #11

There’s nothing wrong with this post factually, but the tone sucks. It has an immensely combative energy for what is not really a charged subject matter. Like sure. Today, a lot of the historical reasons for things seem silly and irrelevant. At one point, they did not seem silly and irrelevant. For compatibility with stuff sticking around from those days, we get some performance penalties that are not strictly necess…

Agreed. I was trying to find the words for what I was so put off by the article, but you nailed it. The tone made me want to disagree with it just by default. Luckily I 1) recognize that I am not qualified to have an opinion on the technical details and 2) Ruthlessly crush instinctual responses until I've thought them through with less emotion. (most of the time... I'm not robot, or perfect) Someone in a sibling thre…

I read the tone as insecurity. Hey look at me, I figured this out. Aren't I smart? All these previous people are dumb. Right? Right!? RIGHT!?!

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#63

Post by throwdbaaway »

blopker wrote: post #38

For what it's worth, Python 3.10 will add -fno-semantic-interposition when built with --with-optimizations by default. However, the slow down only affects Pythons built with --enabled-shared anyway. The issue is discussed here [0]. I also attempted to backport the fix to the official Python Docker image [1], but wasn't able to get much traction. I wish this would land since everyone using these images would get an in…

> However, the slow down only affects Pythons built with --enabled-shared anyway.

On Gentoo, the ebuild for Python and Ruby both have --enable-shared hardcoded.

Also, if you use prebuilt Ruby binaries from RVM, those were also built with --enable-shared. And if you build your own Ruby binaries with ruby-build, it also does --enable-shared.

So, there are still plenty of opportunities around for instant speedup.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#64

Post by smitty1e »

jchw wrote: post #11

There’s nothing wrong with this post factually, but the tone sucks. It has an immensely combative energy for what is not really a charged subject matter. Like sure. Today, a lot of the historical reasons for things seem silly and irrelevant. At one point, they did not seem silly and irrelevant. For compatibility with stuff sticking around from those days, we get some performance penalties that are not strictly necess…

There is a risk of not knowing the historical context for decisions and getting a bit critical about How Daft The Ancients Were.

Much of this is the cruft the author of TFA identifies.

But so much of it was just "nature of the beast" at the time.

Generosity toward these shortcomings is always in order.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#65

Post by kelnos »

CalChris wrote: post #27
jchw wrote: post #11

There’s nothing wrong with this post factually, but the tone sucks. It has an immensely combative energy for what is not really a charged subject matter. Like sure. Today, a lot of the historical reasons for things seem silly and irrelevant. At one point, they did not seem silly and irrelevant. For compatibility with stuff sticking around from those days, we get some performance penalties that are not strictly necess…

The OP was writing about a 29 year old design decision, and he wasn’t writing about a person. Design decisions don’t have feelings. I found his no holds barred clarity about something as obscure as dynamic linking namespaces made for an easier if still not easy read. But that said, I don’t think dynamic linking is in the ELF spec. I believe that’s a de facto OS + dev tools thing rather than an ELF spec de jure thing.…

> I found his no holds barred clarity

I found it not particularly clear at all. The entire thing can be boiled down to 2 or 3 sentences, or maybe as many as 10 if you want to include more background information.

When I was reading it, about halfway through I was thinking "god, when is this mediocre rant going to get to the meat of how to fix the problem?"

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#66

Post by temac »

edit: my comment addresses the text from the original link https://www.facebook.com/dan.colascione/posts/10107358290728...

The author is confused. no-semantic-interposition does not preclude unicity of symbols. There are things like -Wl,-Bsymbolic-functions to relinquish that (and lose strict conformity against the programming language you are using, by the way). When I read a virulent post at least I would like the author to have some deep expertise on the subject, and not mix everything they read about (including related but different matters). I won't even talk about the "interposition is useless anyway because clang has an historical bug" part.

As for elf shared libs providing a single symbol space, there are arguments for and against.

An argument for a single symbol space is that a process with segregated one per dynamic module is a beast the C and C++ standards know nothing about. And don't even get me started on processes with multiple different C or C++ runtimes simultaneously (take a look at the list of CRTs loaded in an instance of explorer.exe, it is horrifying). Another one is that you can easily move things around between dynamic libs or split them, etc. If various subsets are actually used by multiple executables, this can be quite useful (lacking that property, note that MS had to invent their own additional virtual/redirection layer to refactor the Win32 libs)

A practical argument against is that it is "hard" (read: virtually impossible) to dynamically link binary only modules maintained with different-enough toolchains. Likewise for different versions of the same lib (e.g. via transitive deps). But then the question of is this even a good idea shall be asked (well, if you want to load a plugin in a proprietary software I understand this can be wanted -- and this is actually similar to using libs of a proprietary platform; any other use case?)

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#67

Post by hn_throwaway_99 »

fpgaminer wrote: post #18
jchw wrote: post #11

There’s nothing wrong with this post factually, but the tone sucks. It has an immensely combative energy for what is not really a charged subject matter. Like sure. Today, a lot of the historical reasons for things seem silly and irrelevant. At one point, they did not seem silly and irrelevant. For compatibility with stuff sticking around from those days, we get some performance penalties that are not strictly necess…

Not OP, but I read the "antagonistic" style of the post as just the usual catharsis humor. All in-jest. I've used that style of writing plenty before. It's a good way to blow off the steam of working with these rather absurd, archaic systems that we have to tackle on a daily basis. Programming can feel a bit kafkaesque at times, so a bit of aggressive/dark humor goes a long way. But I do agree, it felt too thick. Sti…

The problem I see is that I often see this type of detached, "wow, look at all of these previous shitty decisions!" attitude inject itself, completely unnecessarily, into the workplace.

I've actually made it something I won't compromise on: I refuse to hire people who I suspect will have this attitude. It's one thing to express frustration at previous decisions that make current work more difficult. However, when I see that morph into an arrogance of "how could people have made such a stupid decision" (especially when some of those "people" may still work at the company), without even trying to understand the context of why that decision was made in the first place, it shows to me that person is not an engineer I want to work with.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#68

Post by kelnos »

coldtea wrote: post #57
jordigh wrote: post #44

Earlier quoted context omitted.

Someone wrote that inanimate object. Someone likes that inanimate object. Someone thinks that inanimate object has reasons to be the way it is. Attacking that inanimate object is not without emotional repercussions to those related to that object.

> Someone wrote that inanimate object. Someone likes that inanimate object. You'll find somebody liking everything you can name. Even every attrocity has been done by somebody. Not attacking the person is of course the correct thing. But if you can't even attack the attrocity or bad choice, we've gone too far with this sensitivity thing.

But in this case it's not an atrocity, and whether or not it was even a bad choice (at the time or today) is debatable. It's fine to present an opinion you hold about something, but belittling that thing also tends to implicitly belittle the intellect of people who might agree with that thing, not to mention the people who designed and built that thing.

> But if you can't even attack...

Why do we need to attack something? If we can't explain and support our view that something is wrong or a bad decision without resorting to attacks, perhaps our argument isn't really that strong?

And that's the thing. I don't think the author really presented a strong argument; he tried to convince me by verbally trashing the other side, while the actual logical, coherent argument is buried in a sea of disdain. I think it's still not clear what the default should be. Do we optimize for performance, or for debugability and tinkerability? I mean, I feel like that's one of the classic debates that we still -- and will probably never -- have no hard answer for.

Edited to add: I went back and read the linked Python bug tracker issue[0], which honestly I wish was what HN linked to. It's concise, explains the problem, explains why LD_PRELOAD isn't all that useful for libpython, specifically why this sort of performance degradation is even worse with a library like libpython, and makes sure to call out that this change only affects libpython and not any other shared libraries, where (implicitly) people might find LD_PRELOAD useful.

[0] https://bugs.python.org/issue38980

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#69

Post by beervirus »

codelord wrote: post #34

I would just read the linked post: https://bugs.python.org/issue38980?fbclid=IwAR0cyfahpBywNzbq... As it contains almost the same info without the rant and with better explanation.

And isn’t hosted on Facebook of all things.

Re: Python is 1.3x faster by just adjusting some compiling options for libpython

#70

Post by craftinator »

coldtea wrote: post #55
rtpg wrote: post #46

Earlier quoted context omitted.

This is recalling the old Linus debates, but the aggressiveness _doesn't improve the clarity_, and is basically upping the word count. I'm not tone policing but contesting the premise that "aggressive tone" = "direct". For example >(Windows took a different approach and got it right. In Windows, it's okay for multiple DLLs to provide the same symbol, and there's no sad and desperate effort to pretend that a single na…

> This is recalling the old Linus debates, but the aggressiveness _doesn't improve the clarity_, and is basically upping the word count. It drives the point home though, and keeps the energy levels higher.

I really found my chakras to be stimulated by the wordy venting as well. That's a very good point!
Post reply on HN