Build Tools – Make, no more • Hacker News Board

Re: Build Tools – Make, no more

#41

Post by gyepi » Mon, Apr 21, 2014, 5:56 PM UTC

webjprgm wrote: post #7

apples2apples wrote: post #2

Misses the fundamental point that Make is broken for so many things. To begin with you have to have a single target for each file produced. Generating all the targets to get around this is a nightmare that results in unreadable debug messages and horribly unpredictable call paths. nix tried to solve much of this, but I agree it can't compete with the bazillion other options.

It does not miss it, just ignores it. The author states that there are lots of things we can improve but the point is that we have too many variations on the theme without converging to a solution that has few (or no) dependencies and comes with built-in build knowledge and the ability to discover what you want rather than make you declare it. Such a tool should be: - Zero (or few) dependencies. Likely written in pla…

Some of these requirements should be built into any build tool. However, most can be added easily enough:

For instance, redux [https://github.com/gyepisam/redux] is written in Go (not compiled for binary distribution, but I could add that), is cross platform, supports any mix of languages and tasks, is very easy to learn.

It uses shell scripts to create targets so everything is scriptable. Stuff like recognizing standard folder hierarchies and auto-discovery can be added with small scripts or tools. It can be as simple as you want or as complex as you need.

Re: Build Tools – Make, no more

#42

Post by asb » Mon, Apr 21, 2014, 5:56 PM UTC

I've recently been playing with ninja, which does a good job of not being 'just another make' http://martine.github.io/ninja/. To quote their website, "Where other build systems are high-level languages Ninja aims to be an assembler.". It's used as a backend for GYP (Google Chromium) and is supported by CMake as well. I've had good success generating the files manually using something like ninja_syntax.py: https://github.com/martine/ninja/blob/master/misc/ninja_synt....

I also note that google are working on a successor to GYP, GN which targets Ninja http://code.google.com/p/chromium/wiki/gn.

Re: Build Tools – Make, no more

#43

Post by apples2apples » Mon, Apr 21, 2014, 5:57 PM UTC

exogen wrote: post #19

apples2apples wrote: post #2

Misses the fundamental point that Make is broken for so many things. To begin with you have to have a single target for each file produced. Generating all the targets to get around this is a nightmare that results in unreadable debug messages and horribly unpredictable call paths. nix tried to solve much of this, but I agree it can't compete with the bazillion other options.

Could you post an example of what you mean by the single target/file limitation? As stated I can't tell how implicit rules or a rule to build an entire directory wouldn't be a solution, but maybe I'm not understanding the problem.

Sure, consider a compiler that produces an (foo.o) object file and an annotation (foo.a). Now if a target requires both foo.o and foo.a you have to create two targets on them (even though its really one command).

You can do implicit rules which requires a very verbose makefile, which is what automake and other make generation tools do. God help you figure out what went wrong.

If you make people go to a directory approach you've now imposed a new structure on their code. One reason for the multitude of packages is each one matches their target community better.

Re: Build Tools – Make, no more

#44

Post by demallien » Mon, Apr 21, 2014, 5:58 PM UTC

Heh. The one and only time that I ever wrote a parser in my professional career was for a build tool. In my defence, at the time I didn't know much about command line tools, and had only really programmed in IDEs. So when the new project was to be compiled on the command line, I quickly discovered that maintaining dependencies, changing targets and doing all the other things that a build system generally does by hand quickly gets old. Not knowing that autotools, cmake, ant, and about a bajillion other tools already existed to do just this, I wrote my own language, with a parser in ruby, no less :D

I have since repented. I find autotools (with occasionally a script of [ruby|python|perl] to handle something that would otherwise be tricky to do in make or m4, which is then called by the makefile) works a treat. Just don't try to do anything tricky in the auto tool files - as I said, boot anything exotic out to a separate tool.

Also, any discussion of build tools without also discussing package management is but half a discussion.

Re: Build Tools – Make, no more

#45

Post by drawkbox » Mon, Apr 21, 2014, 5:59 PM UTC

At least we are moving the direction of Grunt/Gulp rather than a maven sort of direction. Many lives lost to maven, somewhat of a Vietnam of build tools. You might think you are a Java developer with it but truly you are a maven servant.

Re: Build Tools – Make, no more

#46

Post by gyepi » Mon, Apr 21, 2014, 6:04 PM UTC

JoshTriplett wrote: post #32

gyepi wrote: post #25

1. Since make has builtin suffix rules, the Makefile could be simplified to: CXX=g++ hello: main.o factorial.o hello.o clean: rm -rf *o hello 2. Shameless plug: he didn't mention redo [1], which is simpler than make and more reliable. The comparable redo scripts to the Makefile would be: cat @all.do redo hello EOF cat hello.do o='main.o factorial.o hello.o' redo-ifchange $o g++ $o -o $3 EOF cat default.o.do redo-ifch…

CXX=g++ isn't necessary either; make already knows about $(CXX) and how to link C++ programs. Also, I think you wanted .o, not o. And compared to that Makefile, the redo scripts you list don't seem simpler at all. I've seen reasonably compelling arguments for redo, but that wasn't one.

> CXX=g++ isn't necessary either; make already knows about $(CXX) and how to link C++ programs.

You're right, of course.

> Also, I think you wanted .o, not o.

I would, yes, but I copied the Makefile ;)

Should have been clearer; I meant that redo is simpler (and more reliable) than make.

For simple projects, redo scripts are a bit longer. However, as the projects grow, the redo scripts reach an asymptote whereas Makefiles don't. The only way to reduce the growth in make is to add functions and implicit rules which get ugly real fast.

Re: Build Tools – Make, no more

#47

Post by rafekett » Mon, Apr 21, 2014, 6:05 PM UTC

the last 10 years in build tools has felt like 1 step forward, two steps back. i like being able to write tasks in any language other than Makefile. however, it seems like many of the new popular options (cake, grunt, etc.) don't do what, to me, is Make's real purpose: resolve dependencies and only rebuild what's necessary. new task runners have either eliminated or pigeonholed the (typically one-to-one in makeland) correspondence between tasks and files, meaning the build system can't be intelligent about what tasks to run and which to not.

computers are fast enough that this doesn't often bother me anymore, but i've run across some huge Rakefiles that could benefit from a rewrite in Make.

Re: Build Tools – Make, no more

#48

Post by joeld42 » Mon, Apr 21, 2014, 6:07 PM UTC

I think everyone goes through a phase where they try to find the perfect build tool, and then at least entertain the idea of writing one themselves.

Eventually, you grow out of it. There's a lot of build tools, each are better at some things than others. It's not that much grunt work to convert things from one to another (even very large projects). If your build tool is working for you, leave it alone. If it's getting in your way or slowing things down, try another one. Move on.

Re: Build Tools – Make, no more

#49

Post by chrismonsanto » Mon, Apr 21, 2014, 6:17 PM UTC

rafekett wrote: post #47

the last 10 years in build tools has felt like 1 step forward, two steps back. i like being able to write tasks in any language other than Makefile. however, it seems like many of the new popular options (cake, grunt, etc.) don't do what, to me, is Make's real purpose: resolve dependencies and only rebuild what's necessary. new task runners have either eliminated or pigeonholed the (typically one-to-one in makeland)…

> however, it seems like many of the new popular options (cake, grunt, etc.) don't do what, to me, is Make's real purpose: resolve dependencies and only rebuild what's necessary.

You might like tup[1]. Its killer feature is that it automatically determines file-based dependencies by tracking reads and writes (using a FUSE filesystem). It has an extreme emphasis on correct, repeatable builds, and is very fast. Other stuff:

- does work in parallel, and will let you know if your build isn't parallel safe. (note it is NOT relying on your specification of dependencies: even if you manually specify dependencies, it will tell you if something's wrong based on what it actually observes your dependencies to be)

- tracks changes to the build script and reruns if the commands change.

- cleans up old output files automatically if build rules are removed.

- lets you maintain multiple build variants (say for different architectures, configurations, etc)

- autogenerates .gitignore files for your build output

- very easy to get started, and "Just Works".

- for advanced usage, it is scriptable in Lua.

I've tried every build system out there. For Unix-y file-based build tasks, tup is, by far, the best. I don't know why it isn't more well known.

[1]: http://gittup.org/tup/index.html

Re: Build Tools – Make, no more

#50

Post by vinkelhake » Mon, Apr 21, 2014, 6:21 PM UTC

I find these posts somewhat amusing. We've got people who (rightfully) question the tools they use and look for alternatives. They then discover Make and have some kind of zen Unix moment that they want to share with the world.

If what you are doing in your flavor-of-the-month build tool translates to a roughly equivalent number of lines in Make, then yes, you should probably look at using Make. But the thing is, Make is stupid, it doesn't know a lot. Sometimes that is a good thing, sometimes it is not.

I've written about this before on HN: I mostly program in C++ and when I build my stuff I want a build tool that understands things like header dependencies, optimization levels, shared libraries etc. It's a bonus if my build files are portable.

My point is that these alternative tools often strive to raise the abstraction level and the reason people use them isn't necessarily because they haven't discovered Make.