Live data from Hacker News

Make Ubuntu packages 90% faster by rebuilding them

gist.github.com

111–120 of 375 posts

Re: Make Ubuntu packages 90% faster by rebuilding them

#111
It's a while since I had to deal with this kind of thing, but my memory was that as soon as you go beyond the flags that the upstream developers use (just to be clear, I mean the upstream developers, not the distro packagers) you're buying yourself weird bugs and a whole lot of indifference if they occur.

I haven't used a non-libc malloc before but I suspect the same applies.

Re: Make Ubuntu packages 90% faster by rebuilding them

#112
post #16

Earlier quoted context omitted.

Sorry, I didn't mean symbols, I meant optimization level. Hilarious that they have symbols stripped, but still outclassed. Love to see why they wouldn't have more optimizations. I'm assuming this isn't the case for more packages? Feels too good.

Debug symbols do not need to be paged in so shouldn’t make much/any difference. Compiling with -O3 can increase code size a lot due to inlining, which can be a little bad.

Mario 64 gets flak for some files not being compiled with optimizations on. There is a YouTuber, whose name I forget at the moment, who has been doing optimization and improvements on it and constantly talks about how this is for the better due to the low instruction cache size and slow speed on the console. Compiling with optimizations explodes code size (unrolling links, inlining, etc) to the point it’s a net loss after factoring in the increased load on the IC.

Re: Make Ubuntu packages 90% faster by rebuilding them

#115
post #16

Earlier quoted context omitted.

Sorry, I didn't mean symbols, I meant optimization level. Hilarious that they have symbols stripped, but still outclassed. Love to see why they wouldn't have more optimizations. I'm assuming this isn't the case for more packages? Feels too good.

> Hilarious that they have symbols stripped, but still outclassed. Debug symbols have 0 runtime penalty, just storage. They're just another section of the binary, referenced by debuggers, and which the loader skips. In any case, all distros break out the symbols into separate files so that they can have their (storage) cake and (debug) eat it too.

Right, I meant the humor to be that folks to not use debug flags, with my naive presumption that they did this as an optimization, only for that not to really help.

Re: Make Ubuntu packages 90% faster by rebuilding them

#116
post #16

Earlier quoted context omitted.

Sorry, I didn't mean symbols, I meant optimization level. Hilarious that they have symbols stripped, but still outclassed. Love to see why they wouldn't have more optimizations. I'm assuming this isn't the case for more packages? Feels too good.

Ubuntu is still building for CPUs with the x86-64-v1 feature set. On experimental images built for x86-64-v3 (roughly the feature set of 10 year old CPUs, a bit newer on low-power CPUs) they have observed 60% improvement on some benchmarks (and much less in some others). Some distros have switched to -v3, Ubuntu is still holding out to support older hardware. The author is compiling to the actual feature set of their…

I was under the, evidently naive, understanding that there were some shenanigans done in the libc to pick optimal code paths based on the architecture. This would obviously have a startup cost on the first run, but presumably the dynamic linking would work to this advantage by having it in a fast state for most calls?

Not sure where I got that idea, though. :(. Will have to look into that later.

Re: Make Ubuntu packages 90% faster by rebuilding them

#117

Misleading title, it's 90% of the faster time. It's about 45% faster. It's actually a little bit interesting, if you are interested in how we use language. You could argue that now you now get 90% more work done in the same amount of time, and that would align with other 'speed' units that we commonly use (miles per hour, words per minute, bits per second). However, the convention in computer performance is to measur…

More misleading is that it implies that all packages can be made 90% faster. This is one particular package.

Re: Make Ubuntu packages 90% faster by rebuilding them

#118
post #16

Earlier quoted context omitted.

Sorry, I didn't mean symbols, I meant optimization level. Hilarious that they have symbols stripped, but still outclassed. Love to see why they wouldn't have more optimizations. I'm assuming this isn't the case for more packages? Feels too good.

Debug symbols do not need to be paged in so shouldn’t make much/any difference. Compiling with -O3 can increase code size a lot due to inlining, which can be a little bad.

Right, my humor there was that I'm assuming people specifically did non-debug flags for the build as an optimization. Only for them to still fall short.

And understood a little on -O3 possibly increasing code size. I had thought that was more of a concern for tight environments than for most systems? Of course, I'd have assumed that -march=native would be more impactful, but the post indicates otherwise.

I said in a top level, but it seems the allocator makes the biggest impact for this application? Would be interesting to see which applications should use different allocators. Would be amazing to see a system where the more likely optimal allocator was default for different applications, based on their typical allocation patterns.

Re: Make Ubuntu packages 90% faster by rebuilding them

#119
If you actually do this, just get Ubuntu to download the exact source package you want. In this case:

    apt-get source jq
Then go into the package and recompile to your heart's content. You can even repackage it for distribution or archiving.

You'll get a result much closer to the upstream Ubuntu, as opposed to getting lots of weird errors and misalignments.

Post reply on HN