Live data from Hacker News

Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

github.com

11–20 of 28 posts

Re: Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

#11
post #9

Can anyone summarize differences with previous work like https://cseweb.ucsd.edu/classes/sp00/cse231/dynamopldi.pdf ?

This is profile-guided optimization: you run your program, collect data on what code runs, and use it to optimize the layout of your program. Dynamo runs optimizations at runtime.

Re: Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

#12

> It has scalability issues and to rewrite a binary with a ~300M text segment size I'm curious what kinds of tasks require a binary with a 300 MB text size…

> I'm curious what kinds of tasks require a binary with a 300 MB text size…

C++ lends itself to bloated binaries. Template instantiations are duplicated even for types where the instantiations could have identical code. Compilers have traditionally favored overly aggressive inlining over whole-program optimization.

Rust didn't solve the problem; it just duplicated the C++ approach, and made it worse by effectively compiling with the equivalent of C++ exceptions enabled. The next great systems language should target binary size and CPU frontend overhead.

Re: Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

#13

> It has scalability issues and to rewrite a binary with a ~300M text segment size I'm curious what kinds of tasks require a binary with a 300 MB text size…

That’s a relatively normal binary size for a ~1 million LOC C++ server from my experience. (statically linked).

Re: Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

#14
post #10

Earlier quoted context omitted.

Yes, I get that; I am curious what goes into a binary with 300 MB of code in it.

Google has very large applications, but also they are using static linking: imagine that you statically link all the possible dependencies in a single binary for each application. 300MB is not considered a large binary at Google actually, I don't know what is public but you may look at Table 2 in this publication for instance: https://www.researchgate.net/publication/314105281_ThinLTO_S... I repro here some data, the…

> they are using static linking

Ah, that would probably be it.

Re: Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

#15

> It has scalability issues and to rewrite a binary with a ~300M text segment size I'm curious what kinds of tasks require a binary with a 300 MB text size…

> I'm curious what kinds of tasks require a binary with a 300 MB text size… C++ lends itself to bloated binaries. Template instantiations are duplicated even for types where the instantiations could have identical code. Compilers have traditionally favored overly aggressive inlining over whole-program optimization. Rust didn't solve the problem; it just duplicated the C++ approach, and made it worse by effectively co…

> Template instantiations are duplicated even for types where the instantiations could have identical code.

compilers and linkers had identical code folding optimizations for a looooong time (20+ years for msvc, a decade or so for GNU gold (--icf option) and GCC (-fipa-icf))

Re: Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

#16

Earlier quoted context omitted.

> I'm curious what kinds of tasks require a binary with a 300 MB text size… C++ lends itself to bloated binaries. Template instantiations are duplicated even for types where the instantiations could have identical code. Compilers have traditionally favored overly aggressive inlining over whole-program optimization. Rust didn't solve the problem; it just duplicated the C++ approach, and made it worse by effectively co…

> Template instantiations are duplicated even for types where the instantiations could have identical code. compilers and linkers had identical code folding optimizations for a looooong time (20+ years for msvc, a decade or so for GNU gold (--icf option) and GCC (-fipa-icf))

I was going to reply with a similar comment, but I'm unable to actually get this to happen when I tried it now: https://godbolt.org/z/4VkEQc. Perhaps the optimization needs more work?

Re: Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

#17

Earlier quoted context omitted.

> I'm curious what kinds of tasks require a binary with a 300 MB text size… C++ lends itself to bloated binaries. Template instantiations are duplicated even for types where the instantiations could have identical code. Compilers have traditionally favored overly aggressive inlining over whole-program optimization. Rust didn't solve the problem; it just duplicated the C++ approach, and made it worse by effectively co…

> Template instantiations are duplicated even for types where the instantiations could have identical code. compilers and linkers had identical code folding optimizations for a looooong time (20+ years for msvc, a decade or so for GNU gold (--icf option) and GCC (-fipa-icf))

> compilers and linkers had identical code folding optimizations for a looooong time (20+ years for msvc, a decade or so for GNU gold (--icf option) and GCC (-fipa-icf))

If you look at the resulting binaries, they don't work as well as you might think.

Re: Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

#18

> It has scalability issues and to rewrite a binary with a ~300M text segment size I'm curious what kinds of tasks require a binary with a 300 MB text size…

> I'm curious what kinds of tasks require a binary with a 300 MB text size… C++ lends itself to bloated binaries. Template instantiations are duplicated even for types where the instantiations could have identical code. Compilers have traditionally favored overly aggressive inlining over whole-program optimization. Rust didn't solve the problem; it just duplicated the C++ approach, and made it worse by effectively co…

Not sure why you get downvoted. I've seen presentations by Ubisoft for example where they replaced templated vectors because of the code bloat which added up too much in their large code base.

Re: Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker

#20

> It has scalability issues and to rewrite a binary with a ~300M text segment size I'm curious what kinds of tasks require a binary with a 300 MB text size…

> I'm curious what kinds of tasks require a binary with a 300 MB text size… C++ lends itself to bloated binaries. Template instantiations are duplicated even for types where the instantiations could have identical code. Compilers have traditionally favored overly aggressive inlining over whole-program optimization. Rust didn't solve the problem; it just duplicated the C++ approach, and made it worse by effectively co…

(That’s only by default, you can panic=abort and not get that overhead too.)
Post reply on HN