Note that if you do this then you will opt out of any security updates not just for jq but also for its regular expression parsing dependency onigurama. For example, there was a security update for onigurama previously; if this sort of thing happens again, you'd be vulnerable, and jq is often used to parse untrusted JSON. > * SECURITY UPDATE: Fix multiple invalid pointer dereference, out-of-bounds write memory corrup…
What about PGO?
Make Ubuntu packages 90% faster by rebuilding them
321–330 of 375 posts
Re: Make Ubuntu packages 90% faster by rebuilding them
#322"Make one Ubuntu package 90% faster by rebuilding it and switching the memory allocator" i wish i could slap people in the face over standard tcp/ip for clickbait. it was ONE package and some gains were not realized by recompilation. i have to give it to him, i have preloaded jemalloc to one program to swap malloc implementation and results have been very pleasant. not in terms of performance (did not measure) but in…
To me it's obviously a scam because there's no way such an improvement can be achieved globally with a single post explanation. 90% faster is a micro-benchmark number.
This is neither a micro-benchmark nor a scam, but it is click-bait by not mentioning jq specifically.
Micro-benchmarks would be testing e.g. a single library function or syscall rather than the whole application. This is the whole application, just not one you might care that much for the performance of.
Other applications will of course see different results, but stuff like enabling LTO, tuning THP and picking a suitable allocator are good, universal recommendations.
Re: Make Ubuntu packages 90% faster by rebuilding them
#323Earlier quoted context omitted.
I thought it would be something like recompiling to utilize AVX512 capabilities or something.
Vector operations like AVX512 will not magically make common software faster. The number of applications that deal with regular operations on large blocks of data is pretty much limited to graphical applications, neural networks and bulk cryptographic operations. Even audio processing doesn't benefit that much from vector operations because a codec's variable-size packets do not allow for efficient vectorization (the…
Vector operations are widely used in common software. Java uses AVX512 for sorting. glibc uses SIMD instructions for string operations.
Re: Make Ubuntu packages 90% faster by rebuilding them
#324Earlier quoted context omitted.
Huh. The IPv6 example is much more confusing that I initially thought. At this point I am entirely unclear as to whether it is actually an example of security through obscurity, regardless of whatever else it might be (a very bad idea to rely on it for one). Rather ironic given that the poster whose claims I was disputing provided it as an example of something that would be universally recognized as such.
I think it’s security through obscurity because in ASLR the randomized base address is a protected secret key material whereas in the ipv6 case it’s unprotected key material (eg every hop between two communicating parties sees the secret). It’s close though which is why IPv6 mapping efforts are much more heuristics based than ipv4 which you can just brute force (along with port #) quickly these days.
I'm finding this semantic rabbit hole surprisingly amusing.
The problem with that line of reasoning is that it implies that data handling practices can determine whether or not a given scheme is security through obscurity. But that doesn't fit the prototypical example where someone uses a super secret and utterly broken home rolled "encryption" algorithm. Nor does it fit the example of someone being careless with the key material for a well established algorithm.
The key defining characteristic of that example is that the security hinges on the secrecy of the blueprints themselves.
I think a case can also be made for a slightly more literal interpretation of the term where security depends on part of the design being different from the mainstream. For example running a niche OS making your systems less statistically likely to be targeted in the first place. In that case the secrecy of the blueprints no longer matters - it's the societal scale analogue of the former example.
I think the IPv6 example hinges on the semantic question of whether a network address is considered part of the blueprint or part of the input. In the ASLR analogue, the corresponding question is whether a function pointer is part of the blueprint or part of the input.
Re: Make Ubuntu packages 90% faster by rebuilding them
#325Earlier quoted context omitted.
I'm genuinely curious what was so undesirable about this sibling comment that it was removed: "ASLR obscures the memory layout. That is security by obscurity by definition. People thought this was okay if the entropy was high enough, but then the ASLR⊕Cache attack was published and now its usefulness is questionable." Usually when a comment is removed, it's pretty obvious why, but in this case I'm really not seeing i…
I believe most people see security through obscurity has an attempt to hide an insecurity. ASLR/KASLR intends to make attackers lives harder by having non consistent offsets of known data structures. Its not obscuring a security flaw, instead its raises an attacks 'single run' effectivness. The ASLR attack that i believe is being referenced is specific to abuse within the browser, and running with a single process. T…
That attack does not require a web browser. The web browser being able to do it showed it was higher severity than you would think than if the proof of concept had been in C, since web browsers run untrusted code all of the time.
Re: Make Ubuntu packages 90% faster by rebuilding them
#326Earlier quoted context omitted.
Portage on top of Fuchsia, probably. There's not all that much more to Gentoo all things considered.
Zircon is miles apart from linux. It’s like saying you’d run Portage on top of Windows. You and what army?
Gentoo Portage had support for running on top of Interix, so it did run on Windows at some point (over 15 years ago).
Re: Make Ubuntu packages 90% faster by rebuilding them
#327Earlier quoted context omitted.
No, that other definition is the incorrect one. Security by obscurity does not require that the attacker is ignorant of the fact you're using it. Say I have an IPv6 network with no firewall, simply relying on the difficulty of scanning the address space. I think that people would agree that I'm using security by obscurity, even if the attacker somehow found out I was doing this. The correct definition is simply "usin…
ASLR is not purely security through obscurity because it is based on a solid security principle: increasing the difficulty of an attack by introducing randomness. It doesn't solely rely on the secrecy of the implementation but rather the unpredictability of memory addresses. Think of it this way - if I guess the ASLR address once, a restart of the process renders that knowledge irrelevant implicitly. If I get your IP…
If someone can guess the randomization within 1 second using the AnC attack, you can restart as much as you want, but it will not improve security.
Re: Make Ubuntu packages 90% faster by rebuilding them
#328what makes me jealous,is not the double performance ,but the fact that he can handle 13000 files in 2 seconds. i am at my ${corporate} job now, and have about 40'000 json files stored locally on the windows laptop. when I need to load a them, it takes minutes . i am still not sure if i read files wrong or NTFS is a pile of trash when comes to reading many smallish files
Try excluding that directory from the virus scanner. Also a dev drive might come in handy for this workload
you were right. it didn't occur to me that antivirus would scan everything I open. Moved all files into an excluded directory, reading files is now almost 10 times faster. Thanks kind stranger !
Re: Make Ubuntu packages 90% faster by rebuilding them
#329Earlier quoted context omitted.
No, I would not agree that you would be using security by obscurity in that example. Not all security that happens to be weak or fragile and involves secret information somewhere is security by obscurity – it’s specifically the security measure that has to be secret. Of course, there’s not a hard line dividing secret information between categories like “key material” and “security measure”, but I would consider ASLR…
I'm inclined to agree and would like to point out that if you take a hardline stance that any reliance on the attacker not knowing something makes it security by obscurity then things like keys become security by obscurity. That's obviously not a useful end result so that can't be the correct definition. It's useful to ask what the point being conveyed by the phrase is. Typically (at least as I've encountered it) it'…
What is missing from these two representations is the ability for something to become trivially bypassable once you know the trick to it. AnC is roughly that for ASLR.
Re: Make Ubuntu packages 90% faster by rebuilding them
#330Earlier quoted context omitted.
Debug symbols do not need to be paged in so shouldn’t make much/any difference. Compiling with -O3 can increase code size a lot due to inlining, which can be a little bad.
Mario 64 gets flak for some files not being compiled with optimizations on. There is a YouTuber, whose name I forget at the moment, who has been doing optimization and improvements on it and constantly talks about how this is for the better due to the low instruction cache size and slow speed on the console. Compiling with optimizations explodes code size (unrolling links, inlining, etc) to the point it’s a net loss…
Kaze Emanuar[0] is the YouTuber you're thinking of.