Could you release a fork of jq whose make does all this fanciness? I use jq A LOT so would love a turnkey solution to make it faster.
Not a fork but a similar program I've been using: https://github.com/tidwall/jj?tab=readme-ov-file#performance
Make Ubuntu packages 90% faster by rebuilding them
41–50 of 375 posts
Re: Make Ubuntu packages 90% faster by rebuilding them
#42-march=native + mimalloc (or jemalloc) should be sufficient without causing significant undefined behavior like -O3 or most extra optimization related compiler arguments.
Re: Make Ubuntu packages 90% faster by rebuilding them
#43Misleading title, it's 90% of the faster time. It's about 45% faster. It's actually a little bit interesting, if you are interested in how we use language. You could argue that now you now get 90% more work done in the same amount of time, and that would align with other 'speed' units that we commonly use (miles per hour, words per minute, bits per second). However, the convention in computer performance is to measur…
I feel like using units of rate (90% faster) and not units of time makes more sense here.
Plus, if you were using units of time, you wouldn’t use the word “faster.” “Takes 45% less time” and “45% faster” are very different assertions, but they both have meaning, both in programming and outside it.
Re: Make Ubuntu packages 90% faster by rebuilding them
#44Gentoo linux is essentially made specifically for people like this, to be able to optimize one’s own linux rig for one’s specific usecase. After initial setup, it’s pretty simple and easy to use, I remember making a ton of friends at matrix’s Gentoo Linux channel, was fun times. https://www.gentoo.org/ Fun fact, initial ChromeOS was basically just custom Gentoo Linux install, I’m not sure if they still use Gentoo Lin…
So ChromeOS and also the OS for GKE are still basically built this way.
Re: Make Ubuntu packages 90% faster by rebuilding them
#45Misleading title, it's 90% of the faster time. It's about 45% faster. It's actually a little bit interesting, if you are interested in how we use language. You could argue that now you now get 90% more work done in the same amount of time, and that would align with other 'speed' units that we commonly use (miles per hour, words per minute, bits per second). However, the convention in computer performance is to measur…
I think you're right. I think you could say the package (as in the code) is 45% faster or that the package increases parsing rate by 90%. But mixing the two is confusing.
Re: Make Ubuntu packages 90% faster by rebuilding them
#46[dead]
Re: Make Ubuntu packages 90% faster by rebuilding them
#47I’m almost more amazed that someone figured out jq’s syntax and got some use out of it. In all seriousness though, are you sure some of this isn’t those blocks being loaded into some kind of file system cache the second and third times? How about if you rebooted and then ran the mimalloc version?
The benchmarking tool being used in this post accounts for that using multiple runs for each invocation, together with a warmup run that is not included in the result metric.
Ah missed that, sorry.
Re: Make Ubuntu packages 90% faster by rebuilding them
#48Earlier quoted context omitted.
If you're going to offer unsupported FUD, I would prefer you direct the FUD at NDEBUG and the fear that it could have disabled a load-bearing assertion.
Load bearing assertions. Even Design by Contract in Eiffel turns off assertions in production.
Just because it's turned off by default doesn't mean it isn't load bearing. If you didn't ever encounter an ASSERT with side effects that were crucial to proper functioning, you just haven't seen enough ASSERT statements.
Rule #1 of programming: If it can go wrong, it will.
Re: Make Ubuntu packages 90% faster by rebuilding them
#49Gentoo linux is essentially made specifically for people like this, to be able to optimize one’s own linux rig for one’s specific usecase. After initial setup, it’s pretty simple and easy to use, I remember making a ton of friends at matrix’s Gentoo Linux channel, was fun times. https://www.gentoo.org/ Fun fact, initial ChromeOS was basically just custom Gentoo Linux install, I’m not sure if they still use Gentoo Lin…
Afaik the Gentoo based ChromeOS is being replaced by Android.
Re: Make Ubuntu packages 90% faster by rebuilding them
#50Surprised it would be 90%. Are the packages all debug symbols? I recall trying with march=native and seeing some improvement, but not enough to care at a system level.
Using preload on the stock Ubuntu binary, to give it mimalloc, they got a 44% speedup.
By rebuilding the binary with different compiler options, but not changing malloc, they got an 20% speedup.
If we naively multiply these speedups, we get 1.78: 78% faster.
How it goes to 1.9 is that when you speed up only the program, but leave malloc the same, malloc matters to its performance a lot more.
When the faster malloc is applied to a program that is compiled better, it will make a better contribution than the 44% seen when the allocator was preloaded into the slower program.
To do the math right, we would have to look into how much time was saved with just the one change, and how much time was saved with the other. If we add those times, and subtract them from the original, slow time, we should get a time that is close to the 1.9 speedup.
Original time: 4.631
Better compiler options alone: 3.853 (-0.778)
Better allocator alone (preload): 3.209 (-1.422)
Add time saved from both: 2.200
Projected time: 4.631 - 2.200 = 2.431
Projected speedup from both: 4.631/2.431 = 1.905
Bang on!