Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Proposal to add build graph output to GNU Make (2020)

jonasdn.blogspot.com

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Proposal to add build graph output to GNU Make (2020)

#21

Post by mhh__ »

eqvinox wrote: post #13

Wouldn't it be possible/easier to create a new/separate script that converts the `make -p` output to the desired `.dot` graph? (Just to have this not be a thing that needs to pass through the GNU make project.) Also I feel like on most nontrivial projects the graph would be gigantic & to get any usability you'd need something interactive where you can collapse/expand the nodes you're (not) interested in. Don't we hav…

It might be easier but having tooling build info your... tooling is genuinely useful

Re: Proposal to add build graph output to GNU Make (2020)

#22

Post by gjvc »

indygreg2 wrote: post #11

I once helped maintain a nearly full featured implementation of make in Python (pymake) that grew out of the Firefox project. As much as I would like to support efforts like this, I feel like it is ultimately doomed to suffer from usability limitations because make does not have a static DAG. Rather, the DAG evolves dynamically as make evaluates targets. There are pesky problems like $(call) and $(eval) where you can…

Does pymake handle colons in filenames any better / easier / etc than GNU make? (I have a task where I have a bunch of files to transform and they have HH:MM:DD timestamps embedded in the basename of the file)

Re: Proposal to add build graph output to GNU Make (2020)

#23

Post by chriswarbo »

saagarjha wrote: post #4

Considering that all build systems must keep a dependency graph internally, it’s surprising how many of them make it incredibly difficult to inspect why things are being rebuilt. I don’t really get it either, I’ve talked to a lot of people and getting feedback on how the build system is doing its task is one of the biggest complaints I see (others are “what is the build system doing” and “why does this have the worst…

FYI you can get this from Nix by giving a .drv file to 'nix-store --query --tree' (a .drv file defines how to build something; it's like an individual Make rule (after all variables, etc. have been evaluated)).

You can get a .drv file using 'nix-instantiate' (or 'nix repl'), e.g.

    $ nix-store --query --tree "$(nix-instantiate -E '(import  {}).bash')"
    /nix/store/378a0jpkxiwzyk6jj6w2y2qdpn8790nc-bash-4.4-p23.drv
    +---/nix/store/j0bqs7l21myz0llnjp9wq73m4nfarzg1-bootstrap-tools.drv
    |   +---/nix/store/71d9qf26s1nzrp34cqw1hh0yrx0h1kbx-cpio.drv
    |   +---/nix/store/7jl6nhfsc0mchccg5w40rfm4zfshpsag-bzip2.drv
    |   +---/nix/store/bci5x35014bbpc87x7ds2n2ffh5syns1-sh.drv
    |   +---/nix/store/fnn1nmssk7fsfn83ik66cxddqxzvnwx6-bootstrap-tools.cpio.bz2.drv
    |   +---/nix/store/p1bhqmlpf6vr9bc5mb99w29mvllcqmsm-mkdir.drv
    |   +---/nix/store/spcnl3whhm392kir1snaar4wlcawr3yq-unpack-bootstrap-tools.sh
    +---/nix/store/04npl9bjcjkpqgwx1s94azbimfgr531v-bash44-012.drv
    |   +---/nix/store/j0bqs7l21myz0llnjp9wq73m4nfarzg1-bootstrap-tools.drv [...]
    |   +---/nix/store/z6aidzdk3yl2k92g8qj8xgzr8glp0vjr-bootstrap-stage0-stdenv-darwin.drv
    |   |   +---/nix/store/1i5y55x4b4m9qkx5dqbmr1r6bvrqbanw-multiple-outputs.sh
    |   |   +---/nix/store/bnj8d7mvbkg3vdb07yz74yhl3g107qq5-patch-shebangs.sh
    |   |   +---/nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
    ...
It can also output other formats, like GraphML and GraphViz. Note that querying a build output will get its runtime dependencies instead of build-time dependencies.

https://man.archlinux.org/man/community/nix/nix-store.1.en#S...

Re: Proposal to add build graph output to GNU Make (2020)

#24

Post by LAC-Tech »

I get the impression that GNU Make is a fairly hard program to update - to the best of my knowledge it still can't handle filenames with spaces in them, for example. So for me it's a lot like a lot of GNU projects - great idea, but a little bit long in the tooth and unlikely to substantially improve at this point.

Re: Proposal to add build graph output to GNU Make (2020)

#27

Post by drothlis »

jhasse wrote: post #9
saagarjha wrote: post #4

Considering that all build systems must keep a dependency graph internally, it’s surprising how many of them make it incredibly difficult to inspect why things are being rebuilt. I don’t really get it either, I’ve talked to a lot of people and getting feedback on how the build system is doing its task is one of the biggest complaints I see (others are “what is the build system doing” and “why does this have the worst…

Like ninja -d explain ?

Ninja's "-d explain" is next to useless in a large build: It says what's dirty but it's hard to relate which dirty input caused which output to be re-built. At the beginning of your build you get pages & pages of "these things are dirty" and then it starts the actual build. Also, this "explain" output is printed before ninja has realised that a lot of those things aren't actually dirty (thanks to "restat").

I've been using this in my build system to make the explain output slightly more useful: https://github.com/ninja-build/ninja/pull/2067

It's just a quick hack that took me half a day (most of which was remembering how to C++ after 8 years...) but I think it makes "-d explain" much more useful.

Really, the "explain" implementation could do with an overhaul. I'd love ninja to serialise the build graph to disk (or maybe just the subset of the graph that actually needed rebuilding). This could be augmented with the reason why each rule was run, and possibly even with the command output for each rule. Then the existing ninja tools like "graph", "browse", and "query" could optionally use this graph instead of the "build.ninja" file, so that you can inspect what happened the last time you ran "ninja".

Realistically I'll never get around to implementing this, and my pull request above gets me close enough.

Re: Proposal to add build graph output to GNU Make (2020)

#28

Post by ReleaseCandidat »

Irix (SGI's) had a make GUI frontend called 'Build Manager' that had a graph view called 'Build Analyzer' of the dependencies. You could even run 'what would happen, if I change this file?' queries on the graph.

https://techpubs.jurassic.nl/manuals/0650/developer/Debugger...

Re: Proposal to add build graph output to GNU Make (2020)

#29

Post by [deleted] »

indygreg2 wrote: post #11

I once helped maintain a nearly full featured implementation of make in Python (pymake) that grew out of the Firefox project. As much as I would like to support efforts like this, I feel like it is ultimately doomed to suffer from usability limitations because make does not have a static DAG. Rather, the DAG evolves dynamically as make evaluates targets. There are pesky problems like $(call) and $(eval) where you can…

[deleted]
Post reply on HN