Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ninja: A simple way to do builds

jvns.ca

Post reply on HN
Print view View original

41–50 of 92 posts

Re: Ninja: A simple way to do builds

#41

Post by brabel »

nsm wrote: post #32

The ninja author has a (relatively) recent blog post about design decisions and lessons learned: * The success and failure of ninja http://neugierig.org/software/blog/2020/05/ninja.html

Thanks for the link, it was a very nice read!

The author wrote Ninja initially as a weekend project. I've written a small build system like that as well, but I think I am the only person using it though :D funny to think that your weekend project could've accidentally become something widespread like Ninja (used to build Chrome, Android, Swift etc).

Re: Ninja: A simple way to do builds

#42

Post by Ericson2314 »

gravypod wrote: post #33

I wish more people were taking a look at Bazel, and the Bazel-family, of build systems. I've looked at Meson/Ninja/Make and it works but it's truely very complicated and very difficult to get hermetic & reproducible builds from these systems. In my opinion, Bazel will be the future once the currently ongoing external dependency management design doc is implemented. I've migrated quite a few builds to bazel at work an…

Bazel is a decent idea but poor execution. I think Nix is the future because it already has the "external dependency" part on lock and it's just a matter of making more fine-grained stuff.

Meson + Nix skipping the Ninja part I would really like to see. I prefer it when the "big dumb builder" and DSLs are developed a bit separately, too. Agreed Ninja's lack of purity is a real bummer.

Re: Ninja: A simple way to do builds

#43

Post by cassepipe »

codemac wrote: post #34
chrismorgan wrote: post #23

Earlier quoted context omitted.

> - if your paths have white space in them This is probably my second biggest complaint about Make. > - if you change your Makefile, nothing will be rebuilt Heh, there’s a common trick for that: just slap Makefile on the end of prerequisite lists. I’ve worked with more complex projects that extend this to versioning the makefile with a `build/.makefile-v${MAKEFILE_VERSION}` common prerequisite so that you can bump th…

> > - if your paths have white space in them > > This is probably my second biggest complaint about Make. If only more of the world used rc instead of bash, we'd be in a better place.

rc?

Re: Ninja: A simple way to do builds

#44

Post by evmar »

chrismorgan wrote: post #23
patrec wrote: post #17

Earlier quoted context omitted.

You are not wrong, but if the world where not full with even worse build systems it'd be a bit saying I prefer to drink from puddles because I find them wherever I go. Make is everywhere, but apart from UX issues such as the dumb syntax or the different flavors you'll encounter, the fundamental problem with it is that it just can't build things properly (the existence of make clean is the dead give away here). That's…

> - if your paths have white space in them This is probably my second biggest complaint about Make. > - if you change your Makefile, nothing will be rebuilt Heh, there’s a common trick for that: just slap Makefile on the end of prerequisite lists. I’ve worked with more complex projects that extend this to versioning the makefile with a `build/.makefile-v${MAKEFILE_VERSION}` common prerequisite so that you can bump th…

Re a new version of inkscape: Ninja does not force fully-correct builds, so it's on the author of the build.ninja file to either model this or not. (This is part of Ninja's objective of not mandating policy.)

To get this fully correct, note that you would need to model all inputs that Inkscape might read, which includes not only binaries but also configuration files or dynamically-loaded plugins. Some systems (I think tup?) track this by tracing executed binaries to see which file system operations they do.

Even with tracing it can be really subtle: for example if a C file has an #include "foo.h", creating a new file named foo.h earlier in the include search path is a meaningful change, which means a file-access-tracing build system also needs to track file-not-found results from previous executions.

(For C programs, also note that getting header dependencies correct includes similarly tracking all the headers in /usr/include that your program transitively includes. See the -MD vs -MMD flags to gcc.)

Re: Ninja: A simple way to do builds

#45

Post by cb321 »

She missed the step where you just say [ "$dst" -nt "$i" ] || svg2pdf "$i" "$dst" in her for-loop. That is about as much extra typing/text as her comment and makes that build script about as fast as alternatives for that simple scenario. { This is not to say there isn't maybe more motivation for grander designs..or various failure modes..just that she missed a step. :-) }

Re: Ninja: A simple way to do builds

#46

Post by pierrebai »

Of note, meson is mentioned to optionally use ninja as a build system, but the same is true of CMake.

On thing to note is that the simplicity of ninja means that a lot of things need to be repeated. For a project I'm working on, the input build.ninja file is 142MB. Just generating that files takes a few minutes.

I think an improvement would be to be able to split it in sub-trees and have ninja produce a "mark" file that has a date that represent when the last time the sub-tree was touched and to not even read any sub-tree files, even the sub-ninja file if an entire sub-tree date is older than the mark file.

Re: Ninja: A simple way to do builds

#47

Post by gravypod »

Ericson2314 wrote: post #42
gravypod wrote: post #33

I wish more people were taking a look at Bazel, and the Bazel-family, of build systems. I've looked at Meson/Ninja/Make and it works but it's truely very complicated and very difficult to get hermetic & reproducible builds from these systems. In my opinion, Bazel will be the future once the currently ongoing external dependency management design doc is implemented. I've migrated quite a few builds to bazel at work an…

Bazel is a decent idea but poor execution. I think Nix is the future because it already has the "external dependency" part on lock and it's just a matter of making more fine-grained stuff. Meson + Nix skipping the Ninja part I would really like to see. I prefer it when the "big dumb builder" and DSLs are developed a bit separately, too. Agreed Ninja's lack of purity is a real bummer.

I'm not so sure about Nix since it seems much more complex in presentation. The meat and potatoes of Bazel is the BUILD langauge. There may be some warts right now but the community is attempting to address these warts. Difficulty around specifying versions of packages, semantics for building containers, etc.

Nix users do not seem to think it's build language is complex. I think Nix might be the worlds best attempt at something like this but it's the same reason why Haskell hasn't gotten adoption: "eeh it looks complicated"

Everyone understands `some_rule(some="argument")` though.

Re: Ninja: A simple way to do builds

#48

Post by barumi »

bachmeier wrote: post #27
chrismorgan wrote: post #13

For comparison, here’s a makefile for the trivial example (though you’ll have to fix the indentation): all: $(patsubst ${things_to_convert},%.svg,%.pdf) %.pdf: %.svg inkscape $ ${things_to_convert} is left undefined, as in the source article. It might be something like $(wildcard [STAR].svg) or $(shell find src -name '[STAR].svg'). [Turn each [STAR] into *; HN formatting idiosyncrasies are in play.] I like makefiles…

> I like makefiles in substantial part because there’s a very high probability of Make already being installed. I’m comfortable enough with them that I didn’t need to look anything up to write any of what I wrote above. There's an even higher probability of Perl being installed, and I'd much rather use that than Make. I've even been known to write a Perl script that calls Make after doing something complex that would…

> There's an even higher probability of Perl being installed

You're showing your ignorance there, champ. Make is one of the utilities that's required by the UNIX specification,which means that each and every single UNIX or UNIX-like OS is required to make Make available to users in order to be considered an UNIX.

So to make this quite explicit, for starters each and every install of macOS ships with make.

Re: Ninja: A simple way to do builds

#49

Post by patrec »

evmar wrote: post #44
chrismorgan wrote: post #23

Earlier quoted context omitted.

> - if your paths have white space in them This is probably my second biggest complaint about Make. > - if you change your Makefile, nothing will be rebuilt Heh, there’s a common trick for that: just slap Makefile on the end of prerequisite lists. I’ve worked with more complex projects that extend this to versioning the makefile with a `build/.makefile-v${MAKEFILE_VERSION}` common prerequisite so that you can bump th…

Re a new version of inkscape: Ninja does not force fully-correct builds, so it's on the author of the build.ninja file to either model this or not. (This is part of Ninja's objective of not mandating policy.) To get this fully correct, note that you would need to model all inputs that Inkscape might read, which includes not only binaries but also configuration files or dynamically-loaded plugins. Some systems (I thin…

> Even with tracing it can be really subtle

The correct way of doing this is not tracing but a hermetic build. If you do a sandboxed build with nix, you will get a specific version of inkscape with a specific hash, which includes will include any inkscape extensions you include. And since the build will only be able to read files in the sandbox, none of the problems you list above can happen -- you can't pull random stuff from the network either.

https://nixos.wiki/wiki/Nix#Sandboxing

Re: Ninja: A simple way to do builds

#50

Post by barumi »

codemac wrote: post #34
chrismorgan wrote: post #23

Earlier quoted context omitted.

> - if your paths have white space in them This is probably my second biggest complaint about Make. > - if you change your Makefile, nothing will be rebuilt Heh, there’s a common trick for that: just slap Makefile on the end of prerequisite lists. I’ve worked with more complex projects that extend this to versioning the makefile with a `build/.makefile-v${MAKEFILE_VERSION}` common prerequisite so that you can bump th…

> > - if your paths have white space in them > > This is probably my second biggest complaint about Make. If only more of the world used rc instead of bash, we'd be in a better place.

> If only more of the world used rc instead of bash,

Make does not use bash. Make uses whatever shell is pointed to by the SHELL env variable. If you don't pass any, it defaults to sh.

Meanwhile, you can use Python in a Makefile for what it's worth.

https://www.gnu.org/software/make/manual/html_node/Choosing-...

Post reply on HN