Live data from Hacker News

Make Ubuntu packages 90% faster by rebuilding them

gist.github.com

191–200 of 375 posts

Re: Make Ubuntu packages 90% faster by rebuilding them

#191

Gentoo linux is essentially made specifically for people like this, to be able to optimize one’s own linux rig for one’s specific usecase. After initial setup, it’s pretty simple and easy to use, I remember making a ton of friends at matrix’s Gentoo Linux channel, was fun times. https://www.gentoo.org/ Fun fact, initial ChromeOS was basically just custom Gentoo Linux install, I’m not sure if they still use Gentoo Lin…

They still use the build system. Woof.

Re: Make Ubuntu packages 90% faster by rebuilding them

#194
post #124
post #38

Note that if you do this then you will opt out of any security updates not just for jq but also for its regular expression parsing dependency onigurama. For example, there was a security update for onigurama previously; if this sort of thing happens again, you'd be vulnerable, and jq is often used to parse untrusted JSON. > * SECURITY UPDATE: Fix multiple invalid pointer dereference, out-of-bounds write memory corrup…

I'm curious how applicable these are, in general? Feels like pointing out that using interior doors in your house misses out on the security afforded from a vault door. Not wrong, but there is also a reason every door in a bank is not a vault door. That is, I don't want to devalue the CVE system; but it is also undeniable that there are major differences in impact between findings?

> I don't want to devalue the CVE system

You could, though. It's 99.9% stuff like this!

Re: Make Ubuntu packages 90% faster by rebuilding them

#195

Earlier quoted context omitted.

If it wasn't for performance, what was gained in using it over something like Slackware and building only the packages you needed to?

USE flags. You can build packages with specific features enabled or disabled, which can further reduce your dependency tree.

For building specific packages with particular flags, wouldn't slackbuilds have been sufficient?

Re: Make Ubuntu packages 90% faster by rebuilding them

#196
post #74

[flagged]

Waste minutes of your time just so anonymous people on the internet can read your comment :). Not everything in life is an optimization problem. Fun little projects like this is what makes us human (and, arguably, are both educational and entertaining).

I see what you're saying but I am also talking about how computers originally saved humans time and that the OP seems to now be slaving away to save the CPU time. Doesn't that seem a little backwards?

If we're talking about micro-seconds of difference, the trade-off doesn't seem worth it. Even on a mass scale where this is somehow adopted, nobody is going to notice the difference. Maybe if this were in something like eCommerce and web browing where the lag translates to profit lost? Or perhaps game engines?

IDK, I just consider humans time more precious than a slow package (that already runs blazingly fast on a CPU that it barely matters.)

Re: Make Ubuntu packages 90% faster by rebuilding them

#197
post #38

Note that if you do this then you will opt out of any security updates not just for jq but also for its regular expression parsing dependency onigurama. For example, there was a security update for onigurama previously; if this sort of thing happens again, you'd be vulnerable, and jq is often used to parse untrusted JSON. > * SECURITY UPDATE: Fix multiple invalid pointer dereference, out-of-bounds write memory corrup…

But isn't there still the kernel of an idea here for a package management system that intelligently decides to build based on platform? Seems like a lot of performance to leave on the table.

Rebuilding from scratch also takes longer than installing a prebuilt package. So while it might be worth it for a heavily used application, in general I doubt it.

Also I think in earlier days the argument to build was so you can optimize the application for the specific capabilities of your system like the supported SIMD instruction set or similar. I think nowadays that is much less of a factor. Instead it would probably be better to do things like that on a package or distribution level (i.e. have one binary distribution package prebuilt by the distribution for different CPU capabilities).

Re: Make Ubuntu packages 90% faster by rebuilding them

#198
post #166

Gentoo linux is essentially made specifically for people like this, to be able to optimize one’s own linux rig for one’s specific usecase. After initial setup, it’s pretty simple and easy to use, I remember making a ton of friends at matrix’s Gentoo Linux channel, was fun times. https://www.gentoo.org/ Fun fact, initial ChromeOS was basically just custom Gentoo Linux install, I’m not sure if they still use Gentoo Lin…

Never seen the HN version of the 'install gentoo' meme before, more sophisticated definitely. > The goal of Gentoo is to have an operating system that builds all programs from source, instead of having pre-built binary packages. While this does allow for advanced speed and customizability, it means that even the most basic components such as the kernel must be compiled from source. It is known through out the Linux c…

Where does that blurb come from, chatgpt? I don't think it's true anymore, last time I checked I think Gentoo had a "normal" liveCD installation for the base system, which you could then recompile on your own if wanted.

Re: Make Ubuntu packages 90% faster by rebuilding them

#199
post #170

Earlier quoted context omitted.

Does ${corporate} prevent you from merging your jsons to a single file with a decent format like parquet?

they don't but that defeats the purpose. i want to be able to inspect the insides, so binary formats are out, i have to easily open & read the content

I don't get how you can't put the content within the merged file and query it but good luck.

Re: Make Ubuntu packages 90% faster by rebuilding them

#200

Gentoo linux is essentially made specifically for people like this, to be able to optimize one’s own linux rig for one’s specific usecase. After initial setup, it’s pretty simple and easy to use, I remember making a ton of friends at matrix’s Gentoo Linux channel, was fun times. https://www.gentoo.org/ Fun fact, initial ChromeOS was basically just custom Gentoo Linux install, I’m not sure if they still use Gentoo Lin…

I had had Gentoo continuously in use since 2003, and only very recently moved off of it (late 2024) when I tried Void Linux. On Void, buildability from source by end users is not a declared goal nor architectural feature, but you have a pretty decent chance of being able to make it work. You can expect one or two hiccups, but if you have decent all-round Linux experience, chances are you'll be able to jump into the build recipes, fix them, make everything work for what you need it to do, and contribute the fixes back upstream. This is what you get from a relentless focus on minimalism and avoiding overengineering of any kind. It's what I had been missing in Gentoo all those years. With Gentoo, I always ended up having to fiddle with use flags and package masks in ways that wouldn't be useful to other users. The build system is so complex that it had been just too difficult for me, over all these years, to properly learn it and learn to fix problems at the root cause level and contribute them upstream. Void should also be an ideal basis for when you don't want to build the entire system from source, but you just want to mix & match distro-provided binaries with packages you've built from source (possibly on the basis of a modified build recipe to better match your needs or your hardware).
Post reply on HN