Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ninja, a small build system with a focus on speed

martine.github.com

Post reply on HN
Print view View original

21–30 of 36 posts

Re: Ninja, a small build system with a focus on speed

#21

Post by groby_b »

julian37 wrote: post #8

Ninja is mostly encoding agnostic, as long as the bytes Ninja cares about (like slashes in paths) are ASCII. This means e.g. UTF-8 or ISO-8859-1 input files ought to work. I was going to say that this is brittle because UTF-8 multi-byte sequences might contain bytes such as 0x2F (forward slash) without actually encoding a slash... but it turns out that's wrong. All bytes in multi-byte sequences always have the high b…

It was actually one of the design goals of UTF-8. As much of a pain as dealing with it is, it certainly had a lot of forethought put in.

Re: Ninja, a small build system with a focus on speed

#22

Post by groby_b »

ajross wrote: post #7

Bravo! Granted, it's yet another make replacement the world really doesn't need . But that said, and unlike all the other attemps, this actually seems to be better than make. Almost always, these things are junk (Ant, I'm looking at you) which at best implement a subset of make's features in a "pure" way (and thus look good to people who don't understand make but like Java-or-whatever). This one actually seems to und…

Speaking as somebody who works on a reasonably large project,yes, the world needs it.

Null build [^1] with cold cache & make: 40s Null build with cold cache & ninja: 12s Null build with hot cache & make: 20s Null build with hot cache & ninja: Ninja saves me 20 seconds every single time I build something. Let's say I kick off about 30-40 builds a day, that's 10-15 minutes each day.

[^1]: I.e. nothing changed

Re: Ninja, a small build system with a focus on speed

#23

Post by apenwarr »

beagle3 wrote: post #10

Anyone has an idea of how this compares to apenwarr's implementation of djb's "redo" concept? Compared to make, redo is extremely simple, yet more versatile, more robust - and potentially very efficient. djb only released the spec (not working code). apenwarr implemented it in Python, which means it's a lot slower than it could be (which you'd mostly feel on nop builds).

The short version is that ninja is declarative while redo is imperative. And I'm not sure that battle will be resolved in my lifetime :) Personally, I prefer imperative stuff most of the time, but lots of smart people disagree.

ninja config files, as I understand it, are designed to be produced by some other tool, because purely-declarative languages are typically a pain for humans to write by hand. So it's one layer in a multi-layer system, hence the integration with cmake.

redo removes layers; you can quite easily write your whole build system in redo, without first translating your configuration from one file type to another. The down side of that design is it's hard to guarantee your build system is "hygienic"; since every .do script is a program, the program might go do things it shouldn't be doing or which might be insecure. In ninja, that sort of thing would be easier to detect/prevent, and in turn it ought to be easy to implement shared caching, distributed builds, etc in a transparent way. It can be very powerful to manipulate declarative structures like ninja's. (Not that you couldn't do those things with redo, but it would be trickier.)

For similar reasons, ninja is probably more portable to Windows than redo is. (redo can run on Windows, but you need a Unix-compatible sh to do it with, which is obviously rather un-Windowsy.)

Re: Ninja, a small build system with a focus on speed

#24

Post by kruhft »

ajross wrote: post #7

Bravo! Granted, it's yet another make replacement the world really doesn't need . But that said, and unlike all the other attemps, this actually seems to be better than make. Almost always, these things are junk (Ant, I'm looking at you) which at best implement a subset of make's features in a "pure" way (and thus look good to people who don't understand make but like Java-or-whatever). This one actually seems to und…

> multiple outputs for a target

I just had to figure out how to do this today with GNU make. It was something like this:

  %Parser.c %Parser.h %Lexer.c %Lexer.h %.tokens: %.g
      antlr $
With the object files required by the main project. Worked like a charm and solved the problem of running antlr multiple times for the produced files during a parallel build.

Re: Ninja, a small build system with a focus on speed

#25

Post by ajross »

groby_b wrote: post #22
ajross wrote: post #7

Bravo! Granted, it's yet another make replacement the world really doesn't need . But that said, and unlike all the other attemps, this actually seems to be better than make. Almost always, these things are junk (Ant, I'm looking at you) which at best implement a subset of make's features in a "pure" way (and thus look good to people who don't understand make but like Java-or-whatever). This one actually seems to und…

Speaking as somebody who works on a reasonably large project,yes, the world needs it. Null build [^1] with cold cache & make: 40s Null build with cold cache & ninja: 12s Null build with hot cache & make: 20s Null build with hot cache & ninja: Ninja saves me 20 seconds every single time I build something. Let's say I kick off about 30-40 builds a day, that's 10-15 minutes each day . [^1]: I.e. nothing changed

Are your make and ninja configurations 1:1? Note that a null build on the kernel (larger than most "reasonably large" projects) is well under 20 seconds.

Re: Ninja, a small build system with a focus on speed

#26

Post by grogs »

julian37 wrote: post #8

Ninja is mostly encoding agnostic, as long as the bytes Ninja cares about (like slashes in paths) are ASCII. This means e.g. UTF-8 or ISO-8859-1 input files ought to work. I was going to say that this is brittle because UTF-8 multi-byte sequences might contain bytes such as 0x2F (forward slash) without actually encoding a slash... but it turns out that's wrong. All bytes in multi-byte sequences always have the high b…

UTF-8 is just... really good.

The only disadvantage I can really think of is that the implementations must be fairly complex. That and determining character count can no longer be calculated from filesize.

Re: Ninja, a small build system with a focus on speed

#27

Post by groby_b »

ajross wrote: post #25
groby_b wrote: post #22

Earlier quoted context omitted.

Speaking as somebody who works on a reasonably large project,yes, the world needs it. Null build [^1] with cold cache & make: 40s Null build with cold cache & ninja: 12s Null build with hot cache & make: 20s Null build with hot cache & ninja: Ninja saves me 20 seconds every single time I build something. Let's say I kick off about 30-40 builds a day, that's 10-15 minutes each day . [^1]: I.e. nothing changed

Are your make and ninja configurations 1:1? Note that a null build on the kernel (larger than most "reasonably large" projects) is well under 20 seconds.

Both generated from the same gyp files, yes. And by "reasonably large", I mean about 9.5MLOC, 30K source files excluding headers :)

Edit: Important info here might be that those times are for an OSX build. I haven't measured, but it seems builds on Linux are faster.

Re: Ninja, a small build system with a focus on speed

#28

Post by evmar »

kruhft wrote: post #24
ajross wrote: post #7

Bravo! Granted, it's yet another make replacement the world really doesn't need . But that said, and unlike all the other attemps, this actually seems to be better than make. Almost always, these things are junk (Ant, I'm looking at you) which at best implement a subset of make's features in a "pure" way (and thus look good to people who don't understand make but like Java-or-whatever). This one actually seems to und…

> multiple outputs for a target I just had to figure out how to do this today with GNU make. It was something like this: %Parser.c %Parser.h %Lexer.c %Lexer.h %.tokens: %.g antlr $ With the object files required by the main project. Worked like a charm and solved the problem of running antlr multiple times for the produced files during a parallel build.

I'm not sure I completely followed your example, but note that the straightforward syntax for multiple outputs from Make does the wrong thing. Here's a discussion of how to make it do what you really want.

http://www.gnu.org/software/automake/manual/html_node/Multip...

Re: Ninja, a small build system with a focus on speed

#29

Post by evmar »

Ralith wrote: post #17
Meai wrote: post #6

I use tup: http://gittup.org/tup/ but I'd like to hear from the author of Ninja what his opinion is on tup.

I also use and enjoy tup, which has the significant features of automatically constructing most of the dependency graph, and proven optimality and correctness for incremental rebuilds. Not to mention that it's highly usable standalone. Does Ninja offer any of that? It doesn't seem to, judging from a skim of the docs.

I (the author of Ninja) think tup is a fine choice for your project.

Ninja was designed to work within a specific pragmatic context: a very large project (Chrome) that had existing constraints on how the build works. (This design also makes Ninja suitable for use from CMake, which means you can use Ninja to build e.g. LLVM in a manner faster than the existing alternatives.)

Here is a longer post on the subject. https://groups.google.com/group/ninja-build/msg/b52e7d3b77bb...

Most projects should probably not use Ninja. A previous iteration of the home page tried to scare users away. I could probably improve that.

Post reply on HN