Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Makefiles – Best Practices

danyspin97.org

Post reply on HN
Print view View original

61–70 of 127 posts

Re: Makefiles – Best Practices

#61

Post by brandmeyer »

chubot wrote: post #25
girzel wrote: post #4

I recently decided it was time to get a better understanding of how makefiles work, and after reading a few tutorials, ended up just reading the manual. It's long, but it's very, very well written (a good example of one of the Gnu projects biggest strengths), to the point where just starting at the top and reading gives an almost tutorial-like effect. Just read the manual!

FWIW I also read the GNU Make manual, and based some code for automatic deps off a profoundly ugly example it had. Then later people on HN showed me a better/simpler way to do it. https://news.ycombinator.com/item?id=15060149 https://www.gnu.org/software/make/manual/html_node/Automatic... After reading the manual and writing 3 substantial Makefiles from scratch, I still think Make is ugly and, by modern standards, no…

Its not actually all that hard to make well-parallelized makefiles, provided you follow a few basic rules.

- Each build step has a unique artifact.

- That artifact is visible to Make as a file in the filesystem.

- That artifact is named $@ in the rule's recipe.

- Every time the recipe is executed, $@ is updated on success.

- If the recipe fails, it must return nonzero to Make.

- All of the dependencies of the artifact are represented in the Makefile

For example, here is how the format checks are run in my current project for some C code. Its mission: To verify that those source files which are under the aegis of clang-format are correctly formatted. BUILD_DIRS is a list of directories containing source code. CFORMATTER is the name of the formatting program. Not everything is under clang-format control, so FORMATTED_SRCS is used to opt-in to it.

BUILD_DIRS_FORMAT = $(addprefix .format-check/,$(BUILD_DIRS))

$(BUILD_DIRS_FORMAT): mkdir -p $@

# There aught to be a better way to control the suffix without becoming a match-anything # rule...

.format-check/%.c: %.c | $(BUILD_DIRS_FORMAT) $(CFORMATTER) $ $@

.format-check/%.h: %.h | $(BUILD_DIRS_FORMAT) $(CFORMATTER) $ $@

# Record the fact that each format check passed by touching a uniquely-named file. # note the call to `false` on error, since `echo` always succeeds.

.format-check/%.diffed: .format-check/% @(diff -u --color=always -- $* $check-formatting: $(addsuffix .diffed,$(addprefix .format-check/, $(FORMATTED_SRCS)))

It's artifacts are:

- A formatted source file for each repository source file

- An empty file in the filesystem for each formatted source file that is identical to the repository source file.

- A tree of directories for the above.

Each format check is run exactly once, and only when source files change. If anything fails, then `make` returns nonzero and the build fails. Its also fully parallelized, since there aren't any neck-down points in the dependency graph. Every one of our pre-commit checks are structured this way. Build verification is as parallel as possible for fresh builds. Engineers can resolve and verify their problems quickly and incrementally when they fail.

Re: Makefiles – Best Practices

#62

Post by vbernat »

> Using the assignment operator = will instead override CC and LDD values from the environment; it means that we choose the default compiler and it cannot be changed without editing the Makefile.

People can use `make CC=clang` to override assignments. That's a pretty common use.

Re: Makefiles – Best Practices

#63

Post by wahern »

chubot wrote: post #55
jgrahamc wrote: post #45

Earlier quoted context omitted.

GNU Make makes it easy to write a makefile that works well for a clean serial build, but has bugs once you add -j or when your repository is in some intermediate state. So true. I wrote about this and solutions here: https://www.cmcrossroads.com/article/pitfalls-and-benefits-g...

Thanks, I bought your GNU Make book a couple years ago and read pretty much the whole thing! Along with reading the GNU Make manual a few years before that, I made an attempt to "give Make a fair shake". I liked it, but it's honestly weird to me that a book published in 2015 can improve on the state of the understanding of a tool with 40 years of heritage :) I also get this "groundhog day" effect after about 10 years…

  > 1) A ninja file for fast, parallel, incremental developer
  > builds
  >
  > 2) A shell script for portable builds for distro
  > packagers/end users. This is just a clean serial build, so
  > it can be a shell script rather than a Makefile.
If the developer isn't regularly using the same build that downstream users are, the build for downstream will be perpetually broken.

Re: Makefiles – Best Practices

#64

Post by jschwartzi »

klodolph wrote: post #32
chubot wrote: post #25

Earlier quoted context omitted.

FWIW I also read the GNU Make manual, and based some code for automatic deps off a profoundly ugly example it had. Then later people on HN showed me a better/simpler way to do it. https://news.ycombinator.com/item?id=15060149 https://www.gnu.org/software/make/manual/html_node/Automatic... After reading the manual and writing 3 substantial Makefiles from scratch, I still think Make is ugly and, by modern standards, no…

> In my mind, the biggest problem is that it offers you virtually no help in writing correct parallel and incremental builds. That’s interesting, because in my mind, parallel and incremental builds are the main features of Make, the features that it is best at, and all other features are secondary. It sounds like your problem is with the correctness part. Make gives you no tools to enforce that your build rules are a…

As I recall, the guy who developed Make wrote it in a weekend.

Re: Makefiles – Best Practices

#65

Post by chubot »

wahern wrote: post #63
chubot wrote: post #55

Earlier quoted context omitted.

Thanks, I bought your GNU Make book a couple years ago and read pretty much the whole thing! Along with reading the GNU Make manual a few years before that, I made an attempt to "give Make a fair shake". I liked it, but it's honestly weird to me that a book published in 2015 can improve on the state of the understanding of a tool with 40 years of heritage :) I also get this "groundhog day" effect after about 10 years…

> 1) A ninja file for fast, parallel, incremental developer > builds > > 2) A shell script for portable builds for distro > packagers/end users. This is just a clean serial build, so > it can be a shell script rather than a Makefile. If the developer isn't regularly using the same build that downstream users are, the build for downstream will be perpetually broken.

It's no different than CMake generating Makefiles or Visual Studio projects, or autoconf generating Makefiles, which is the state of the art in open source.

I'm not saying you write them by hand -- you generate them from the same build description, and the generator can preserve some invariants. It should basically do a topological sort ahead of time rather than at runtime.

The more likely source of breakage is that the user's environment is different, i.e. they don't have a particularly library installed). So even if you choose pure GNU make, you still have that source of breakage, and you generally should test for it. I test my shell in a chroot with different versions of libc and without GNU readline. I need that even though I'm using pure GNU make at the moment.

Re: Makefiles – Best Practices

#66

Post by rcfox »

SamWhited wrote: post #23

I really wish blog posts like this would at least mention that they are going to use GNUMake extensions. There's some good stuff in here, but just calling it "make" and leaving it at that is misleading. It's not going to work on my minimal (non-GNU) Linux boxes that mostly run NetBSD make (bmake in many distros) or busybox style tools. I know a very small minority of people do something like that (or maybe not, what…

I doubt most people even know that there are different versions. (I didn't!)

Re: Makefiles – Best Practices

#67

Post by int_19h »

scottlocklin wrote: post #51
nrclark wrote: post #40

GNU Make is an awesome task-runner. I use it all the time, for all sorts of things. It's my shell-script replacement, and I write one-off Makefiles frequently. It's got a nice macro-processor, some helpful string manipulation tools, and expresses task dependencies perfectly. The manual is also very well written. But if you're writing a software package, consider using CMake, Meson, Autotools, or something similar. Un…

I haven't found a reason to not just use Make yet when starting from green fields, including some very big projects. In fact, I have often been annoyed at some projects picking something like CMake for no reason beyond it's "more advanced," but which ends up just being an extra dependency I have to fetch and install. If I were to pick one of the above for building large projects on linux variants, which one would you…

If simpler is better, then why not Redo over Make?

https://redo.readthedocs.io/en/latest/

Re: Makefiles – Best Practices

#68

Post by int_19h »

kstenerud wrote: post #34

I've been using makefiles for decades, and finally decided to try something more modern. It turns out that CMake has come a LOOOONG way, and is almost nice to use, aside from the fact that it's horribly complicated. But with some basic templates to work from, it's actually pretty easy to get a project started. https://github.com/kstenerud/modern-cmake-templates

Take a look at Meson, too.

Re: Makefiles – Best Practices

#69

Post by LukeShu »

jgrahamc wrote: post #14
girzel wrote: post #4

I recently decided it was time to get a better understanding of how makefiles work, and after reading a few tutorials, ended up just reading the manual. It's long, but it's very, very well written (a good example of one of the Gnu projects biggest strengths), to the point where just starting at the top and reading gives an almost tutorial-like effect. Just read the manual!

Yes. It is highly recommended that you read that book (buy it to support the FSF). If you want more I wrote a book on GNU Make ( https://nostarch.com/gnumake ) which takes things further than the GNU Make manual. A large amount of the content of the book came from a sequence of blog posts on GNU Make by me: https://blog.jgc.org/2013/02/updated-list-of-my-gnu-make-art...

I don't think the FSF is currently selling physical copies of the Make manual.

Re: Makefiles – Best Practices

#70

Post by phyllostachys »

shrimp_emoji wrote: post #35
kstenerud wrote: post #34

I've been using makefiles for decades, and finally decided to try something more modern. It turns out that CMake has come a LOOOONG way, and is almost nice to use, aside from the fact that it's horribly complicated. But with some basic templates to work from, it's actually pretty easy to get a project started. https://github.com/kstenerud/modern-cmake-templates

Or just use Rust. :3 Cargo handles makes with very simple TOML scripts, and it's the more modern language, to boot.

Every time I start a little thing in C or C++ and I write a corresponding Makefile I get annoyed by it and write the thing in Rust. Cargo is really nice. I've considered using a Cargo.toml and build.rs file just to build C and C++ files before.
Post reply on HN