Live data from Hacker News

Make Ubuntu packages 90% faster by rebuilding them

gist.github.com

201–210 of 375 posts

Re: Make Ubuntu packages 90% faster by rebuilding them

#201
Reading this, I wonder a few things that seem distro quick wins:

1) Why don't distros replace the glibc allocator with one of the better ones?

2) Why don't distros allow for making server-specific builds for only a few packages? You don't have to pay the compilation cost for everything, just a list of 2 or 3 packages.

Re: Make Ubuntu packages 90% faster by rebuilding them

#202

Earlier quoted context omitted.

Agree for most short running apps. I updated my comment to reflect issues with apps that are constantly reallocating, and running for longer that 60 seconds. But you are absolutely correct for most short running apps, 99% recommended to replace glibc. However, there is an app or two where glibc stability doesnt trigger a pathological use cases, and you have no choice. Hence why its the default, since there are less c…

Looked at your updated post and it looks like you’re operating under wildly incorrect assumptions. 1. Fragmentation: MIMalloc and the newest TCMalloc definitely handle this better than glibc. This is well established in many many many benchmarks. 2. In terms of process lifetime, MIMalloc (Microsoft Cloud) and TCMalloc (Google Cloud) are designed to be run for massive long-lived services that continually allocate/deal…

Challenge yourself to produce some numbers first. If there are many many many benchmarks it shouldn't be too difficult to link one. Just saying something is "well established" doesn't really help without some other context.

Re: Make Ubuntu packages 90% faster by rebuilding them

#203

Reading this, I wonder a few things that seem distro quick wins: 1) Why don't distros replace the glibc allocator with one of the better ones? 2) Why don't distros allow for making server-specific builds for only a few packages? You don't have to pay the compilation cost for everything, just a list of 2 or 3 packages.

Glibc is a reliable generalist while most other ones specialise in something: single thread, multiple threads, slab, arena style, etc ..

Some of the other things disabled might be useful for distros in general, like ndebug assertions or compiling with debig symbols.

Re: Make Ubuntu packages 90% faster by rebuilding them

#204
Make a specific ubuntu package 90% faster by rebuilding it after changing the memory allocator, that is.

Should probably work for Debian and RedHat too. For this particular package.

Edit: based just on the title i initially thought this is an article about turning Ubuntu into Linux From Scratch.

Re: Make Ubuntu packages 90% faster by rebuilding them

#205
post #4

> What happens if you grab the jq source code from Launchpad, then configure and rebuild it with no flags at all? Even that is about 2-4% faster than the Ubuntu binary package. Why wouldn't that be identical?

Because instead of using the Debian build, I just used `./configure; make`

How does that explain it?

Re: Make Ubuntu packages 90% faster by rebuilding them

#206
post #123
post #111

It's a while since I had to deal with this kind of thing, but my memory was that as soon as you go beyond the flags that the upstream developers use (just to be clear, I mean the upstream developers, not the distro packagers) you're buying yourself weird bugs and a whole lot of indifference if they occur. I haven't used a non-libc malloc before but I suspect the same applies.

I've been building my own emacs for a long time, and have yet to hit any weird bugs. I thought that as long as you avoid any unsafe optimizations, you should be fine? Granted, I also thought that -march=native was the main boost that I was seeing. This post indicates that is not necessarily the case. I also suspect that any application using floats is more likely to have rough edges?

Did you try -ffast-math? IIRC that used to break emacs in some subtle way, while providing no extra speed.

Re: Make Ubuntu packages 90% faster by rebuilding them

#207
Reading that such a simple change can get such a big speedup, my first thought is to let the authors of jq know. Maybe there's a caveat to be aware of, maybe they'll test it and end up making it faster for everyone. Useful to drop a quick note pretty much no matter the result, I think?

The article doesn't seem to even consider that option and I don't see any comment here mentioning this either. Am I missing something?

Re: Make Ubuntu packages 90% faster by rebuilding them

#209

Earlier quoted context omitted.

If it wasn't for performance, what was gained in using it over something like Slackware and building only the packages you needed to?

USE flags. You can build packages with specific features enabled or disabled, which can further reduce your dependency tree.

Reducing the dependency tree gets a bit more complicated once you consider that now you have to satisfy not only runtime dependencies for all packages but also build-time dependencies. There may be ways of cleaning that up after a build, but next time you want to emerge a new package you'll just end up having to re-build the build-time dependencies, so in practice you'll just end up leaving them there. There is an ability to emerge packages to a separate part of the filesystem tree (ROOT="/my/chroot" emerge bla), so that you have one build-time system act as a kind of incubator for a runtime system that gets to be minimal. But you'll end up encountering problems that most other Gentoo users wouldn't encounter, having to do with the separation between build-time dependencies and runtime dependencies not being correctly made in the recipes. Personally, I had been relying on this feature for roughly the last 10 years, but there has been steady deterioration there over the years and I eventually gave up late last year.

Re: Make Ubuntu packages 90% faster by rebuilding them

#210
post #120

Earlier quoted context omitted.

Portage on top of Fuchsia, probably. There's not all that much more to Gentoo all things considered.

Zircon is miles apart from linux. It’s like saying you’d run Portage on top of Windows. You and what army?

Windows was built with a POSIX subsystem from the very beginning. WSL 1 uses something analogous.

https://en.m.wikipedia.org/wiki/Microsoft_POSIX_subsystem

The Zircon kernel does not support signals, so basic C is not going to work well.

"It is heavily inspired by Unix kernels, but differs greatly. For example, it does not support Unix-like signals, but incorporates event-driven programming and the observer pattern."

https://en.m.wikipedia.org/wiki/Fuchsia_(operating_system)#K...

Post reply on HN