Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I made a real-time C/C++/Rust build visualizer

danielchasehooper.com

Post reply on HN
Print view View original

61–70 of 91 posts

Re: I made a real-time C/C++/Rust build visualizer

#61

Post by flakes »

entelechy wrote: post #31

Love it! We did something similar using strace/dtruss back in 2018 with https://buildinfer.loopperfect.com/ and were generating graphs (using eg. graphviz and perfetto.dev) and BUCK files on the back of that Whilst we regrettably never came around to package it as a propper product, we found it immensly valuable in our consulting work, to pinpoint issues and aid the conversion to BUCK/Bazel. We used graphviz, https:/…

Curious, what were you using for doing syscall logging? LD_PRELOAD tricks, or ebpf filtering?

Re: I made a real-time C/C++/Rust build visualizer

#62

Post by shakna »

DiddlyWinks wrote: post #11
dhooper wrote: post #9

Earlier quoted context omitted.

Thank you! Yeah it can be used for any type of program, but I haven't been able to think of anything besides compilation that creates enough processes to be interesting. I'm open to ideas!

Video encoding and 3-D rendering are a couple that come to mind; I'd think they'd launch quite a few. This looks like a really cool tool!

Just as a random example in the area, I had a project where I transformed every frame in a video, using a custom binary, before encoding them back to the video.

Hundreds of thousands of processes were normal.

Re: I made a real-time C/C++/Rust build visualizer

#63

Post by ethan_smith »

tiddles wrote: post #17

Nice, I’ve been looking for something like this for a while. I’ve noticed on my huge catkin cmake project that cmake is checking the existence of the same files hundreds of times too. Is there anything that can hook into fork() and provide a cached value after the first invocation?

You could try ccache with the CCACHE_SLOPPINESS=file_stat_matches option, or implement a filesystem-level caching proxy like CachingFS or FUSE-based solutions that intercept and cache those redundant stat() calls.

Re: I made a real-time C/C++/Rust build visualizer

#65

Post by hagendaasalpine »

Night_Thastus wrote: post #10

I am extremely interested in this. I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible. It's also a bit of a messy build with steps like copying a bunch of files from the source into the build folder. Multiple languages (C, C++, Fortran, Python), custom cmake steps, etc. If this tool can handle that…

Tsoding wrote https://github.com/tsoding/nob.h, single header C library for cross platform builds, only requirement is cc. GDB profiling tools can then be used to look at your build step. It's a neat idea. I suspect this is not an option but Nix is great build tool if you are dealing with multiple languages.

Re: I made a real-time C/C++/Rust build visualizer

#67

Post by fransje26 »

Night_Thastus wrote: post #10

I am extremely interested in this. I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible. It's also a bit of a messy build with steps like copying a bunch of files from the source into the build folder. Multiple languages (C, C++, Fortran, Python), custom cmake steps, etc. If this tool can handle that…

> I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible.

I have a similar problem, with a tangential question that I think about from time to time without really having the time to investigate it further, unfortunately.

I notice sometimes that CMake recompiles files that shouldn't have been affected by the code changes made previously. Like recompiling some independent objects after only slight changes to a .cpp file without any interface changes.

So I often wonder if CMake is not making some file more inter-dependent than what they are, leading to longer compilation times.

Re: I made a real-time C/C++/Rust build visualizer

#68

Post by jppittma »

hagendaasalpine wrote: post #65
Night_Thastus wrote: post #10

I am extremely interested in this. I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible. It's also a bit of a messy build with steps like copying a bunch of files from the source into the build folder. Multiple languages (C, C++, Fortran, Python), custom cmake steps, etc. If this tool can handle that…

Tsoding wrote https://github.com/tsoding/nob.h , single header C library for cross platform builds, only requirement is cc. GDB profiling tools can then be used to look at your build step. It's a neat idea. I suspect this is not an option but Nix is great build tool if you are dealing with multiple languages.

Btw, he has a YouTube channel and streams. I recommend it if you’re seeking imposter syndrome.

Re: I made a real-time C/C++/Rust build visualizer

#70

Post by epage »

bvisness wrote: post #43
dhooper wrote: post #4

Earlier quoted context omitted.

(OP here) Thanks! My call with the Mozilla engineer was cut short, so we didn't have time to go into detail about what he found, I want to look into it myself.

Hello, I am the engineer in question. I am not actually super familiar with the details of the build system, but from when I saw, the main issues were: - Lots of constant-time slowness at the beginning and end of the build - Dubious parallelism, especially with unified builds - Cargo being Cargo Overall it mostly looks like a soup of `make` calls with no particular rhyme or reason. It's a far cry from the ninja examp…

What `cargo being cargo` problems are you having?
Post reply on HN