Live data from Hacker News

Ninja: A simple way to do builds

jvns.ca

21–30 of 92 posts

Re: Ninja: A simple way to do builds

#21
post #12

I don't know I did a few things in Chromium and admittedly it's a really big project BUT ninja and autoninja get really cryptic and complex pretty fast. To be fair it's better than make or the older gyp setup BUT for non trivial workloads ninja is pretty fast but leaves a lot to be desired in terms of DX.

Are you sure that is a Ninja issue and not an issue with Chromium's build generator GN?

Re: Ninja: A simple way to do builds

#22
The one thing that's keeping me from using Ninja for the use-case Julia is describing is lacking support for reading environment variables in build.ninja. I understand why that's not part of Ninja, but it does stunt its usefulness.

Re: Ninja: A simple way to do builds

#23
post #17

For comparison, here’s a makefile for the trivial example (though you’ll have to fix the indentation): all: $(patsubst ${things_to_convert},%.svg,%.pdf) %.pdf: %.svg inkscape $ ${things_to_convert} is left undefined, as in the source article. It might be something like $(wildcard [STAR].svg) or $(shell find src -name '[STAR].svg'). [Turn each [STAR] into *; HN formatting idiosyncrasies are in play.] I like makefiles…

You are not wrong, but if the world where not full with even worse build systems it'd be a bit saying I prefer to drink from puddles because I find them wherever I go. Make is everywhere, but apart from UX issues such as the dumb syntax or the different flavors you'll encounter, the fundamental problem with it is that it just can't build things properly (the existence of make clean is the dead give away here). That's…

> - if your paths have white space in them

This is probably my second biggest complaint about Make.

> - if you change your Makefile, nothing will be rebuilt

Heh, there’s a common trick for that: just slap Makefile on the end of prerequisite lists. I’ve worked with more complex projects that extend this to versioning the makefile with a `build/.makefile-v${MAKEFILE_VERSION}` common prerequisite so that you can bump the version if the change is such that you’ll need to rebuild everything.

> - if inkscape writes out a partial file because full disk and you fix the problem, the partial svg will no be rebuilt

Or the more common form, you run a fallible program and it produces errors but still produced a file (e.g. if you used piping, `> $@` instead of a `--output=$@` command line argument that only creates the file after processing has succeeded).

I think “build errored but produced a file” is probably my biggest complaint about Make. You can work around it in a few ways; `|| (touch --date=@0 $@; false)` is probably my favourite, which I used in https://chrismorgan.info/blog/make-and-git-diff-test-harness....

> - if you install a new version of inkscape nothing will be rebuilt

I’m curious: would Ninja rebuild on a new version of Inkscape? I’d be pleasantly surprised and view it much more favourably if it does.

In the scope of Make, prerequisite satisfaction is all reckoned with timestamps, which is a thing I’m not always fond of, so you definitely can’t do this properly. I think the closest you could get is generating a file containing the versions of all your prerequisites, and arrange for it to only be updated when they’ve changed. Basically a slight variant of the MAKEFILE_VERSION trick discussed earlier, though definitely much more involved, especially if you want it to have one filename always rather than just being a hash of all the versions.

And yes, all this is supporting your point that Make is not the best thing in this space, which I quite agree with. But for simple things I reckon it’s often still worth using despite this.

Re: Ninja: A simple way to do builds

#24
post #4
post #2

That's just BASH with extra steps.

The advantage are the incremental builds they mention. If just one input file changes and you run ninja again, it'll only re-run one step instead of converting all files again.

I'm pretty sure make does that too. The problem is that most people don't know how to write a proper Makefile, even for a simple project, so make ends up doing more work than it should. For small projects, I've tried ninja a while ago, and frankly the effort seemed unjustified, since (a) I already could do the same things with make, and (b) make is already available on most unix-likes.

Ninja seems to be better suited for big projects, where make's slowness in dependency resolution shows. Ninja files also look easier to generate automatically.

Re: Ninja: A simple way to do builds

#25

If you don't know it, CMake can use ninja instead of make for compilation so instead of "mkdir build && cd build && cmake .. && make" you can run "mkdir build && cd build && cmake -GNinja && ninja". It feels faster on my side projects (haven't benchmarked it really), but most importantly, when running a parallel build it does not interleave the output of the different steps, making it much easier to read warnings and…

A small warning for code with many CMake ExternalProject dependencies: the defaults for cmake+ninja will recursively run `ninja -jN` for each level of the dependency tree, which can spawn a very large number of compile jobs.

There is a proposal to add jobserver support for global process count management [1], but it has not been merged for philosophical reasons (IIRC). The CMake folks maintain a soft fork with this patch applied, available in the `ninja` package on PyPI and probably elsewhere.

[1] https://github.com/ninja-build/ninja/issues/1139

Re: Ninja: A simple way to do builds

#26
post #22

The one thing that's keeping me from using Ninja for the use-case Julia is describing is lacking support for reading environment variables in build.ninja. I understand why that's not part of Ninja, but it does stunt its usefulness.

There's an obvious fix for that... Mix in some M4... ;-)

https://en.m.wikipedia.org/wiki/M4_(computer_language)

Re: Ninja: A simple way to do builds

#27

For comparison, here’s a makefile for the trivial example (though you’ll have to fix the indentation): all: $(patsubst ${things_to_convert},%.svg,%.pdf) %.pdf: %.svg inkscape $ ${things_to_convert} is left undefined, as in the source article. It might be something like $(wildcard [STAR].svg) or $(shell find src -name '[STAR].svg'). [Turn each [STAR] into *; HN formatting idiosyncrasies are in play.] I like makefiles…

> I like makefiles in substantial part because there’s a very high probability of Make already being installed. I’m comfortable enough with them that I didn’t need to look anything up to write any of what I wrote above.

There's an even higher probability of Perl being installed, and I'd much rather use that than Make. I've even been known to write a Perl script that calls Make after doing something complex that would be "not fun" to add to a Makefile.

Re: Ninja: A simple way to do builds

#28

If you don't know it, CMake can use ninja instead of make for compilation so instead of "mkdir build && cd build && cmake .. && make" you can run "mkdir build && cd build && cmake -GNinja && ninja". It feels faster on my side projects (haven't benchmarked it really), but most importantly, when running a parallel build it does not interleave the output of the different steps, making it much easier to read warnings and…

Ninja is especially beneficial on large projects with lots of source files. Where Make can take 30+ seconds to run a "no-op" build, when nothing needs rebuilding, ninja does it instantly.

Re: Ninja: A simple way to do builds

#29

Earlier quoted context omitted.

Ninja hasn't been created for small one-liners like in this blog post, but for massive C++ builds with tens- or hundreds-of-thousands of source files arranged in a complex dependency tree. Figuring out quickly what needs to be rebuilt for massive builds like this isn't trivial, but that's essentially the one important thing that ninja does. Also see: https://ninja-build.org/manual.html#_design_goals

... What mechanism is used to perform incremental builds? It is referenced in the article and unexplained from what I can tell. What enables "quickly" for you to determine what needs to be built? Make simply compares timestamps I believe. ... What mechanisms/tricks speed up builds? Or is it just the "simplicity" that is the BFD? Edit: It sure would have been helpful if somewhere was stated: "the build system of Chrom…

I'm not sure it's the responsibility of others to ensure you have enough information not to look silly in a hasty post ;)

Re: Ninja: A simple way to do builds

#30

If you don't know it, CMake can use ninja instead of make for compilation so instead of "mkdir build && cd build && cmake .. && make" you can run "mkdir build && cd build && cmake -GNinja && ninja". It feels faster on my side projects (haven't benchmarked it really), but most importantly, when running a parallel build it does not interleave the output of the different steps, making it much easier to read warnings and…

Ninja is especially beneficial on large projects with lots of source files. Where Make can take 30+ seconds to run a "no-op" build, when nothing needs rebuilding, ninja does it instantly.

The conclusion at https://david.rothlis.net/ninja-benchmark/ (2016) was that this doesn't have an impact until you get to really large projects (assuming you are able to utilise make -j).
Post reply on HN