Live data from Hacker News

I made a real-time C/C++/Rust build visualizer

danielchasehooper.com

41–50 of 91 posts

Re: I made a real-time C/C++/Rust build visualizer

#41
post #40

Earlier quoted context omitted.

When I was trying to improve compile time for my game engine, I ended up using compiled size as a proxy measure. Although it is an imperfect correlation, the fact that compiled size is deterministic across build runs and even across builds on different machines makes it easier to work with than wall clock time.

Wait, this is not intuitive at all for me. If the compiler is working harder wouldn't that result in a more compact binary? Maybe I'm thinking too much from an embedded software POV. I suppose the compiler does eventually do IO but IO isn't really the constraint most of the time right?

While you can cause the compiler to run longer to squeeze the binary size down, the compiler has a baseline number of compiler passes that it runs over the IR of the program being compiled. These compiler passes generally take time proportional to the input IR length, so a larger program takes longer to compile. Most compiler passes aren't throwing away huge amounts of instructions (dead code elimination being a notable exception, but the analysis to figure out which pieces of dead code can be eliminated still is operating on the input IR). So it's not a perfect proxy, but in general, if the output of your compiler is 2MB of code, it probably took longer to process all the input and spit out that 2MB than if the output of your compiler was 200KB.

Re: I made a real-time C/C++/Rust build visualizer

#42
post #40

Earlier quoted context omitted.

When I was trying to improve compile time for my game engine, I ended up using compiled size as a proxy measure. Although it is an imperfect correlation, the fact that compiled size is deterministic across build runs and even across builds on different machines makes it easier to work with than wall clock time.

Wait, this is not intuitive at all for me. If the compiler is working harder wouldn't that result in a more compact binary? Maybe I'm thinking too much from an embedded software POV. I suppose the compiler does eventually do IO but IO isn't really the constraint most of the time right?

Of course there are the cases where a huge template structure with complex instantiation and constexpr code compiles down to a single constant, but for most parts of the code I would assume there is a proportion from code size, via compile time to binary size.

Re: I made a real-time C/C++/Rust build visualizer

#43
post #4
post #2

That's really cool. Fascinating to think about all the problems that get missed due to poor or missing visualizations like this. I did a lot of work to improve the Mozilla build system a decade ago where I would have loved this tool. Wish they would have said what problem they found.

(OP here) Thanks! My call with the Mozilla engineer was cut short, so we didn't have time to go into detail about what he found, I want to look into it myself.

Hello, I am the engineer in question. I am not actually super familiar with the details of the build system, but from when I saw, the main issues were:

- Lots of constant-time slowness at the beginning and end of the build

- Dubious parallelism, especially with unified builds

- Cargo being Cargo

Overall it mostly looks like a soup of `make` calls with no particular rhyme or reason. It's a far cry from the ninja example the OP showed in his post.

Re: I made a real-time C/C++/Rust build visualizer

#45
Really cool tool, but perhaps not for the original use-case. I often find myself trying to figure out what call tree a large Bash script creates, and this looks like it visualises it well.

This would have been really useful 6 months ago, when I was trying to figure out what on earth some Jetson tools actually did to build and flash an OS image.

Re: I made a real-time C/C++/Rust build visualizer

#46
Suggestion to the blog author - put:

> Here it is recording the build of a macOS app:

>

At the top of the page, it should be right under the header.

You made a thing, so show the thing. You can waffle on about it later. Just show the thing.

Re: I made a real-time C/C++/Rust build visualizer

#48

I am extremely interested in this. I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible. It's also a bit of a messy build with steps like copying a bunch of files from the source into the build folder. Multiple languages (C, C++, Fortran, Python), custom cmake steps, etc. If this tool can handle that…

Can you set CC=time gcc ?

Re: I made a real-time C/C++/Rust build visualizer

#50

I am extremely interested in this. I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible. It's also a bit of a messy build with steps like copying a bunch of files from the source into the build folder. Multiple languages (C, C++, Fortran, Python), custom cmake steps, etc. If this tool can handle that…

It's not "nearly impossible" but actually built in: https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdo... For the actual compile time you can easily insert a wrapper script. To be honest I haven't done that in over 4 years, but it has been done by many and it is easy.

There may be times when CMake itself is the bottleneck but it is almost certainly an issue with your dependencies and so on. CMake has many features to assist you in speeding up your compile and link time too. But it would take a series of blog posts to describe how you should try to speed it up.

Post reply on HN