Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ninja: A simple way to do builds

jvns.ca

Post reply on HN
Print view View original

31–40 of 92 posts

Re: Ninja: A simple way to do builds

#31

Post by CUViper »

antoinealb wrote: post #6

If you don't know it, CMake can use ninja instead of make for compilation so instead of "mkdir build && cd build && cmake .. && make" you can run "mkdir build && cd build && cmake -GNinja && ninja". It feels faster on my side projects (haven't benchmarked it really), but most importantly, when running a parallel build it does not interleave the output of the different steps, making it much easier to read warnings and…

You can also use "cmake --build " to build regardless of which generator was used.

Re: Ninja: A simple way to do builds

#33

Post by gravypod »

I wish more people were taking a look at Bazel, and the Bazel-family, of build systems. I've looked at Meson/Ninja/Make and it works but it's truely very complicated and very difficult to get hermetic & reproducible builds from these systems. In my opinion, Bazel will be the future once the currently ongoing external dependency management design doc is implemented. I've migrated quite a few builds to bazel at work and I've seen a dramatic reduction in compile times - not even from compiling source code! Doing things like packaging zips, generating code, generating docs (swagger), etc all adds up in CI time and automating, and caching, absolutely all of it is astonishingly useful.

Re: Ninja: A simple way to do builds

#34

Post by codemac »

chrismorgan wrote: post #23
patrec wrote: post #17

Earlier quoted context omitted.

You are not wrong, but if the world where not full with even worse build systems it'd be a bit saying I prefer to drink from puddles because I find them wherever I go. Make is everywhere, but apart from UX issues such as the dumb syntax or the different flavors you'll encounter, the fundamental problem with it is that it just can't build things properly (the existence of make clean is the dead give away here). That's…

> - if your paths have white space in them This is probably my second biggest complaint about Make. > - if you change your Makefile, nothing will be rebuilt Heh, there’s a common trick for that: just slap Makefile on the end of prerequisite lists. I’ve worked with more complex projects that extend this to versioning the makefile with a `build/.makefile-v${MAKEFILE_VERSION}` common prerequisite so that you can bump th…

> > - if your paths have white space in them

>

> This is probably my second biggest complaint about Make.

If only more of the world used rc instead of bash, we'd be in a better place.

Re: Ninja: A simple way to do builds

#35

Post by drothlis »

chrismorgan wrote: post #13

For comparison, here’s a makefile for the trivial example (though you’ll have to fix the indentation): all: $(patsubst ${things_to_convert},%.svg,%.pdf) %.pdf: %.svg inkscape $ ${things_to_convert} is left undefined, as in the source article. It might be something like $(wildcard [STAR].svg) or $(shell find src -name '[STAR].svg'). [Turn each [STAR] into *; HN formatting idiosyncrasies are in play.] I like makefiles…

> Ninja looks like it tracks dependencies roughly automatically, whereas dependencies are manual in Make

Ninja has built-in support for the ".d" files generated by gcc's "-M" family of flags, but you still have to make your gcc command-line actually pass the right flags, and you have to tell Ninja the name of the ".d" file. See https://lwn.net/Articles/706404/

Another neat feature of Ninja is that it's much better at "pruning" downstream targets if you rebuilt something (because of an out-of-date timestamp) but then it turned out to be bit-for-bit identical as it was before. In this case, downstream targets that depend on this intermediate target don't need to be rebuilt.

Re: Ninja: A simple way to do builds

#36

Post by patrec »

chrismorgan wrote: post #23
patrec wrote: post #17

Earlier quoted context omitted.

You are not wrong, but if the world where not full with even worse build systems it'd be a bit saying I prefer to drink from puddles because I find them wherever I go. Make is everywhere, but apart from UX issues such as the dumb syntax or the different flavors you'll encounter, the fundamental problem with it is that it just can't build things properly (the existence of make clean is the dead give away here). That's…

> - if your paths have white space in them This is probably my second biggest complaint about Make. > - if you change your Makefile, nothing will be rebuilt Heh, there’s a common trick for that: just slap Makefile on the end of prerequisite lists. I’ve worked with more complex projects that extend this to versioning the makefile with a `build/.makefile-v${MAKEFILE_VERSION}` common prerequisite so that you can bump th…

> I’m curious: would Ninja rebuild on a new version of Inkscape? I’d be pleasantly surprised and view it much more favourably if it does.

I haven't used ninja, but nix and (unless memory fails me) bazel derived build systems will. Nix basically has a completely hermetic world, and everything is referred to by hash that includes the recursive dependency tree.

Re: Ninja: A simple way to do builds

#37

Post by dcolkitt »

gravypod wrote: post #33

I wish more people were taking a look at Bazel, and the Bazel-family, of build systems. I've looked at Meson/Ninja/Make and it works but it's truely very complicated and very difficult to get hermetic & reproducible builds from these systems. In my opinion, Bazel will be the future once the currently ongoing external dependency management design doc is implemented. I've migrated quite a few builds to bazel at work an…

Out of curiosity, how difficult did you find the migration process? Especially in terms of having to restructure the project.

I have no direct experience, but from what I've heard Bazel is very opinionated about structure. Which makes it particularly challenging to migrate pre-existing projects that may not conform to its view. But again this all hearsay, so I'd like to hear from someone with direct experience.

Re: Ninja: A simple way to do builds

#38

Post by gravypod »

dcolkitt wrote: post #37
gravypod wrote: post #33

I wish more people were taking a look at Bazel, and the Bazel-family, of build systems. I've looked at Meson/Ninja/Make and it works but it's truely very complicated and very difficult to get hermetic & reproducible builds from these systems. In my opinion, Bazel will be the future once the currently ongoing external dependency management design doc is implemented. I've migrated quite a few builds to bazel at work an…

Out of curiosity, how difficult did you find the migration process? Especially in terms of having to restructure the project. I have no direct experience, but from what I've heard Bazel is very opinionated about structure. Which makes it particularly challenging to migrate pre-existing projects that may not conform to its view. But again this all hearsay, so I'd like to hear from someone with direct experience.

It was pretty easy for the software I was migrating because it was already in a monorepo and we were already using some very similar conventions. Anyone writing Java code that uses Gradle or Maven shouldn't have much pain migrating. I'm acutally currently supporting both Gradle and Bazel on the same backend code base to give another team (who depends on my code) a chance to gracefully migrate.

The "hardest" thing to migrate has been C and Python code. For these the package for something is based on an absolute file path from the root of your repo. So `company/foobar/thing.py` is `from company.foobar import thing`.

I find this very useful personally because it's now standardized through the codebase (no one just plops a setup.py somewhere and does whatever they want) but it makes the migration difficult because you need to change the paths.

There's some work being done in rules_python and in cc_library/cc_binary that lets you control this behavior and hack it to align with your current company standards though which is really nice to use for a gradual migration.

Re: Ninja: A simple way to do builds

#39

Post by slezyr »

gravypod wrote: post #33

I wish more people were taking a look at Bazel, and the Bazel-family, of build systems. I've looked at Meson/Ninja/Make and it works but it's truely very complicated and very difficult to get hermetic & reproducible builds from these systems. In my opinion, Bazel will be the future once the currently ongoing external dependency management design doc is implemented. I've migrated quite a few builds to bazel at work an…

For same no one looks at the GYP, GN, and now Bazel. Google bury their build tools just as fast as other products.

And something about Java dependency.

Re: Ninja: A simple way to do builds

#40

Post by niccl »

gravypod wrote: post #33

I wish more people were taking a look at Bazel, and the Bazel-family, of build systems. I've looked at Meson/Ninja/Make and it works but it's truely very complicated and very difficult to get hermetic & reproducible builds from these systems. In my opinion, Bazel will be the future once the currently ongoing external dependency management design doc is implemented. I've migrated quite a few builds to bazel at work an…

Do you know whether/how it can deal with database tables dependent on source .csv files?

My job involves lots of ETL-type work, where the extracted source file(s) for a particular table might have names that include the extract date or version, and a table might depend on other tables. I've been looking at make for dealing with the process, but automatic dependency management seems hard, and dealing with the variable source file names even harder (possibly because I'm not more than a make novice).

Post reply on HN