Live data from Hacker News

Make Ubuntu packages 90% faster by rebuilding them

gist.github.com

281–290 of 375 posts

Re: Make Ubuntu packages 90% faster by rebuilding them

#281

Earlier quoted context omitted.

strict-aliasing, which is what caused that bug to manifest, is enabled at O2.

Yep, it's what caused the bug to manifest, but who knows if that UB would have caused -O2 optimizations to mangle the result as well. EDIT: first one is -funswitch-loops, though

Probably would have. There's no shortage of UB bugs revealed by O2 after all, including major security issues.

Re: Make Ubuntu packages 90% faster by rebuilding them

#282
post #190

He is still missing profile guided optimizations (with his testcase), and bolt. And in some/most cases -Os is faster than -O3 Boehm GC is usually slower than glibc with lots of allocs. MPS would be better, but needs lots of rewriting. https://github.com/Ravenbrook/mps

BOLT is only good for ~2% in this scenario, which is about what I would have expected from glancing at the profiles.

-Os is much slower, as I would have expected. Nothing in the perf data suggested high L1i miss rate. The .text of the "size optimized" program is, for some reason, 16x larger.

Re: Make Ubuntu packages 90% faster by rebuilding them

#283
post #212
post #202

Earlier quoted context omitted.

Challenge yourself to produce some numbers first. If there are many many many benchmarks it shouldn't be too difficult to link one. Just saying something is "well established" doesn't really help without some other context.

https://github.com/microsoft/mimalloc?tab=readme-ov-file#per...

Oddly enough I've written about those results before:

https://www.dropbox.com/scl/fi/evnn6yoornh9p6l7nq1t9/Irrepro...

Re: Make Ubuntu packages 90% faster by rebuilding them

#284

Earlier quoted context omitted.

That’s why it’s a bad idea to use one allocator for everything in existence . It’s terrible that everyone pays the cost of thread safety even for single threaded applications - or even multithreaded applications with disciplined resource management.

While I do agree with the general sentiment, I think the default should be to use the safer ones that can handle multi threaded usage. If someone wants to use a bad allocator like glibc that doesn't handle concurrency well, then they should certainly be free to switch.

So now you’re using std vector and it’s taking a lock for no reason, even though push_back on the same vector across threads isn’t thread safe to begin with.

Haphazard multithreading is not a sane default.

I understand a million decisions have been made so that we can’t go flip that switch back off, but we’ve got to learn these lessons for the future.

Re: Make Ubuntu packages 90% faster by rebuilding them

#286

"Make one Ubuntu package 90% faster by rebuilding it and switching the memory allocator" i wish i could slap people in the face over standard tcp/ip for clickbait. it was ONE package and some gains were not realized by recompilation. i have to give it to him, i have preloaded jemalloc to one program to swap malloc implementation and results have been very pleasant. not in terms of performance (did not measure) but in…

I thought it would be something like recompiling to utilize AVX512 capabilities or something.

Vector operations like AVX512 will not magically make common software faster. The number of applications that deal with regular operations on large blocks of data is pretty much limited to graphical applications, neural networks and bulk cryptographic operations. Even audio processing doesn't benefit that much from vector operations because a codec's variable-size packets do not allow for efficient vectorization (the main exception being multi-channel effects processing as used in DAW).

Re: Make Ubuntu packages 90% faster by rebuilding them

#287
post #160
post #99

Earlier quoted context omitted.

I'm willing to believe that will execute in less than 2 seconds, but it doesn't work as given.

Can you post a link to the JSON file? Try this (the placement of FROM was incorrect): ch "WITH arrayJoin(features) AS f SELECT f.properties.SitusCity FROM 'a.json' WHERE f.properties.TotalNetValue

It's a public data file accessible at https://data.acgov.org/datasets/2b026350b5dd40b18ed7a321fdcd...

Re: Make Ubuntu packages 90% faster by rebuilding them

#288
post #3

Surprised it would be 90%. Are the packages all debug symbols? I recall trying with march=native and seeing some improvement, but not enough to care at a system level.

It's not a 90% speedup, it's ~50% (still quite impressive). The author seems to be confused, because the original jq is 1.9x slower than the optimized one.

That depends on how you're representing the speedup.

To travel 10 miles, at 60 MPH, takes 10 minutes. Make it 100% faster, at 120 MPH, and that time becomes 5 minutes. Travel just as far in 50% of the time. Or travel just as far 100% faster. The 90% speedup matches the reduction of the time it takes to nearly half (a 90% (projected) speedup, or about a 45% time reduction, as mathed out by kazinator `Projected speedup from both: 4.631/2.431 = 1.905`). Your claim that its closer to 50% is correct from a total time taken perspective, just coming at it from the other direction.

Re: Make Ubuntu packages 90% faster by rebuilding them

#289
post #206
post #123

Earlier quoted context omitted.

I've been building my own emacs for a long time, and have yet to hit any weird bugs. I thought that as long as you avoid any unsafe optimizations, you should be fine? Granted, I also thought that -march=native was the main boost that I was seeing. This post indicates that is not necessarily the case. I also suspect that any application using floats is more likely to have rough edges?

Did you try -ffast-math? IIRC that used to break emacs in some subtle way, while providing no extra speed.

I thought touching the math optimizations directly was in the "unsafe" bucket. Really the only optimization I was aiming for was -march=native. That and the features like native compilation that have made it to the release.

I do think I saw improvements. But I never got numbers, so I'm assuming most of my feel was wishful thinking. Reality is a modern computer is hella fast for something like emacs.

I did see compilation mode improve when I trimmed down the regexes it watches for to only the ones I knew were relevant for me. That said, I think I've stopped doing that; so maybe that is a lot better?

Re: Make Ubuntu packages 90% faster by rebuilding them

#290
post #123

Earlier quoted context omitted.

I've been building my own emacs for a long time, and have yet to hit any weird bugs. I thought that as long as you avoid any unsafe optimizations, you should be fine? Granted, I also thought that -march=native was the main boost that I was seeing. This post indicates that is not necessarily the case. I also suspect that any application using floats is more likely to have rough edges?

Complex software usually has some undefined behavior lurking that at higher or even just different optimization levels can trigger the compiler to do unexpected things to the code. It happens all the time in my line of work. If there's an extensive test suite you can run to verify that it still works mostly as expected then it's easier.

This is one where I suspect we don't disagree. But "all the time" can have a very different feel between people.

It also used to happen that just changing processors was likely to find some problems in the code. I have no doubt that still happens, but I'd also expect it has reduced.

Some of this has to be a convergence on far fewer compilers than we used to encounter. I know there are still many c compilers. Seems there are only two common ones, though. Embedded, of course, is a whole other bag of worms.

Post reply on HN