Live data from Hacker News

Make Ubuntu packages 90% faster by rebuilding them

gist.github.com

141–150 of 375 posts

Re: Make Ubuntu packages 90% faster by rebuilding them

#141
post #114

Earlier quoted context omitted.

-O3 doesn't cause undefined behavior.

No, but it can often expose undefined behavior that doesn't stop a program from "working" on lower optimization levels. Which is why I'm in support of always building your own software with -O3.

Are you sure about that? I’m not aware of any additional UB exploitation enabled by O3 vs O2 or Os.

Re: Make Ubuntu packages 90% faster by rebuilding them

#142

Earlier quoted context omitted.

No, that other definition is the incorrect one. Security by obscurity does not require that the attacker is ignorant of the fact you're using it. Say I have an IPv6 network with no firewall, simply relying on the difficulty of scanning the address space. I think that people would agree that I'm using security by obscurity, even if the attacker somehow found out I was doing this. The correct definition is simply "usin…

No, I would not agree that you would be using security by obscurity in that example. Not all security that happens to be weak or fragile and involves secret information somewhere is security by obscurity – it’s specifically the security measure that has to be secret. Of course, there’s not a hard line dividing secret information between categories like “key material” and “security measure”, but I would consider ASLR…

I'm inclined to agree and would like to point out that if you take a hardline stance that any reliance on the attacker not knowing something makes it security by obscurity then things like keys become security by obscurity. That's obviously not a useful end result so that can't be the correct definition.

It's useful to ask what the point being conveyed by the phrase is. Typically (at least as I've encountered it) it's that you are relying on secrecy of your internal processes. The implication is usually that your processes are not actually secure - that as soon as an attacker learns how you do things the house of cards will immediately collapse.

Re: Make Ubuntu packages 90% faster by rebuilding them

#143
post #64
post #49

Earlier quoted context omitted.

Afaik the Gentoo based ChromeOS is being replaced by Android.

It will be interesting if Gentoo supported Fuchsia next.

Gentoo already has prefix support for non-Linux systems and used to have at least some interest in a full Gentoo/kFreeBSD, so it's plausible.

Re: Make Ubuntu packages 90% faster by rebuilding them

#144

Earlier quoted context omitted.

Mimalloc is a general purpose allocator like JEMalloc / TCMalloc. Glibc is known to have a pretty bad allocator that modern allocators like MIMalloc & the latest TCMalloc (not the one available by default in Ubuntu) run laps around. While of course speedup may be variable, generally the benchmarks show an across the board speedup (whether that matters for any given application is something entirely different). As for…

Agree for most short running apps. I updated my comment to reflect issues with apps that are constantly reallocating, and running for longer that 60 seconds. But you are absolutely correct for most short running apps, 99% recommended to replace glibc. However, there is an app or two where glibc stability doesnt trigger a pathological use cases, and you have no choice. Hence why its the default, since there are less c…

Looked at your updated post and it looks like you’re operating under wildly incorrect assumptions.

1. Fragmentation: MIMalloc and the newest TCMalloc definitely handle this better than glibc. This is well established in many many many benchmarks.

2. In terms of process lifetime, MIMalloc (Microsoft Cloud) and TCMalloc (Google Cloud) are designed to be run for massive long-lived services that continually allocate/deallocate over long periods of time. Indeed, they have much better system behavior in that allocating a bunch of objects & then freeing them actually ends up eventually releasing the memory back to the OS (something glibc does not do).

> However, there is an app or two where glibc stability doesnt trigger a pathological use cases, and you have no choice.

I’m going to challenge you to please produce an example with MIMalloc or the latest TCMalloc (or heck - even any real data point from some other popular allocators vs vague anectodes). This just simply is not something these allocators suffer from and would be major bugs the projects would solve.

Re: Make Ubuntu packages 90% faster by rebuilding them

#145
post #26

I’m almost more amazed that someone figured out jq’s syntax and got some use out of it. In all seriousness though, are you sure some of this isn’t those blocks being loaded into some kind of file system cache the second and third times? How about if you rebooted and then ran the mimalloc version?

jq has point-free programming. it's intuitive once you wrap your head around it. See this: https://en.wikipedia.org/wiki/Tacit_programming#jq

Thank you! That's helpful. I got the examples in that Wikipedia section to run using the `--null-input` flag:

  $ jq --null-input '[1, 2] | add'
  3

Re: Make Ubuntu packages 90% faster by rebuilding them

#146

Misleading title, it's 90% of the faster time. It's about 45% faster. It's actually a little bit interesting, if you are interested in how we use language. You could argue that now you now get 90% more work done in the same amount of time, and that would align with other 'speed' units that we commonly use (miles per hour, words per minute, bits per second). However, the convention in computer performance is to measur…

After thinking about it more, I think I know why it seems misleading. They are talking about the change in the larger value as a % of the smaller value. That's what is misleading. They are saying it is reduced by 90% (of some other thing). When you say "We reduced THING by N%" it's just assumed that the N% is N% of THING, not OTHER THING.

Re: Make Ubuntu packages 90% faster by rebuilding them

#147

Earlier quoted context omitted.

No, that other definition is the incorrect one. Security by obscurity does not require that the attacker is ignorant of the fact you're using it. Say I have an IPv6 network with no firewall, simply relying on the difficulty of scanning the address space. I think that people would agree that I'm using security by obscurity, even if the attacker somehow found out I was doing this. The correct definition is simply "usin…

ASLR is not purely security through obscurity because it is based on a solid security principle: increasing the difficulty of an attack by introducing randomness. It doesn't solely rely on the secrecy of the implementation but rather the unpredictability of memory addresses. Think of it this way - if I guess the ASLR address once, a restart of the process renders that knowledge irrelevant implicitly. If I get your IP…

I don't like that example because the damaged cause by and the difficulty of recovering from a secret leaking is not what determines the classification. There exist keys that if leaked would be very time consuming to recover from. That doesn't make them security by obscurity.

I think the key feature of the IPv6 address example is that you need to expose the address in order to communicate. The entire security model relies on the attacker not having observed legitimate communications. As soon as an attacker witnesses your system operating as intended the entire thing falls apart.

Another way to phrase it is that the security depends on the secrecy of the implementation, as opposed to the secrecy of one or more inputs.

Re: Make Ubuntu packages 90% faster by rebuilding them

#150
post #69

Earlier quoted context omitted.

ASLR is not security through obscurity though. It forces attacker to get a pointer leak before doing almost anything (even arbitrary read and arbitrary write primitives are useless without a leak with ASLR). As someone with a bit of experience in exploit dev, it makes a world of a difference and is one of the most influential hardenings, next to maybe stack cookies and W^X.

ASLR is by definition security through obscurity. That doesn't make it useless, as there's nothing wrong with using obscurity as one layer of defenses. But that doesn't change what it fundamentally is: obscuring information so that an attacker has to work harder.

Security by obscurity is keeping your security algorithms and design secret, not your data at runtime secret.
Post reply on HN