Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Tutorial on Portable Makefiles

nullprogram.com

Post reply on HN
Print view View original

61–70 of 114 posts

Re: A Tutorial on Portable Makefiles

#61

Post by wahern »

0x09 wrote: post #7

The problem comes in as soon as you need conditionals, which is likely when attempting to build something portably. There may be some gymnastics that can be done to write around the lack of their presence in standard make, but otherwise your options are: - Supply multiple makefiles targeting different implementations - Bring in autotools in all its glory (at this point you are depending on an external GNU package any…

POSIX Make supports conditionals, thanks to recursive macro expansion. And while POSIX doesn't yet support a macro construct to invoke and capture shell utilities, you can use both the GNU $(shell COMMAND) and Sun $(COMMAND:sh) syntax. All the BSDs and Solaris support the latter, and GNU Make the former. Both constructs are ignored with an empty string expansion where they're not supported, and no implementation supports both.

To see what I'm talking about, see my proof of concept library that I've put together.

https://github.com/wahern/autoguess/blob/config-guess/config...

Re: A Tutorial on Portable Makefiles

#62

Post by chubot »

rossy wrote: post #57
chubot wrote: post #47

Earlier quoted context omitted.

Don't you need a pattern rule for %.d? GNU Make doesn't seem to come with this rule. There is a profoundly ugly example here: https://www.gnu.org/software/make/manual/html_node/Automatic... %.d: %.c @set -e; rm -f $@; \ $(CC) -M $(CPPFLAGS) $ $@.$$$$; \ sed 's,\($*\)\.o[ :]*,\1.o $@ : ,g' $@; \ rm -f $@.$$$$ I copied it into a running example here for anyone curious: https://github.com/oilshell/blog-code/tree/master/…

If you have -MD or -MMD in your CFLAGS, GCC (and Clang) will generate the .d files during compilation without you having to add anything else to the Makefile, and without requiring ugly sed magic.

I just tried this, and it seems wrong on two counts:

1) The sed magic is required for adding the dependencies of the .d file itself. For example, if the timestamp of your header changes, it may have acquired new dependencies, and a new .d file has to be generated BEFORE make determines if the .c file is out of date.

See:

https://www.gnu.org/software/make/manual/html_node/Automatic...

The purpose of the sed command is to translate (for example):

    main.o : main.c defs.h
into:

    main.o main.d : main.c defs.h
The first line isn't correct because the .d file itself has no dependencies.

2) By the time you are compiling, it's too late to generate .d. The .d files are there to determine IF you need to compile.

EDIT: I am trying to generate a test case that shows this fails, but it seems to actually work.

Hm yes I'm convinced it works, but I have to think about why. I guess one way of saying it is that the previous .d file is always correct. Hm.

Re: A Tutorial on Portable Makefiles

#63

Post by caf »

tedunangst wrote: post #10
0x09 wrote: post #7

The problem comes in as soon as you need conditionals, which is likely when attempting to build something portably. There may be some gymnastics that can be done to write around the lack of their presence in standard make, but otherwise your options are: - Supply multiple makefiles targeting different implementations - Bring in autotools in all its glory (at this point you are depending on an external GNU package any…

Another possibility is conditional compilation at the C level. thefile.c #includes thefile_unix.c or thefile_windows.c as appropriate.

That doesn't really help situations like "iconv is in libiconv on some systems, and is in libc on others" though.

Re: A Tutorial on Portable Makefiles

#64

Post by thetic »

> The bad news is that inference rules are not compatible with out-of-source builds. You’ll need to repeat the same commands for each rule as if inference rules didn’t exist. This is tedious for large projects, so you may want to have some sort of “configure” script, even if hand-written, to generate all this for you. This is essentially what CMake is all about. That, plus dependency management.

This isn't a case for CMake. It's a case against POSIX Make. The proposed "portability" and "robustness" of adherence to the POSIX standard are not worth hamstringing the tool. GNU Make is ubiquitous and is leaps and bounds ahead of pure Make.

Re: A Tutorial on Portable Makefiles

#65

Post by rossy »

chubot wrote: post #62
rossy wrote: post #57

Earlier quoted context omitted.

If you have -MD or -MMD in your CFLAGS, GCC (and Clang) will generate the .d files during compilation without you having to add anything else to the Makefile, and without requiring ugly sed magic.

I just tried this, and it seems wrong on two counts: 1) The sed magic is required for adding the dependencies of the .d file itself. For example, if the timestamp of your header changes, it may have acquired new dependencies, and a new .d file has to be generated BEFORE make determines if the .c file is out of date. See: https://www.gnu.org/software/make/manual/html_node/Automatic... The purpose of the sed command is…

(2) is not quite correct. The old .d file from the previous compilation is actually all you need to determine whether the .c file needs to be recompiled. It works in all cases. If the .c file is new (or you're doing a clean rebuild of the whole project,) it will always be compiled, because there will be no corresponding .o. If the .c file, or any of the .h files in the old .d gain new header dependencies, they must have been modified, so their timestamps will be newer than the .o file from the last build, hence the .c file will be recompiled and a new up-to-date .d file will be generated (because a new .d file is always generated when the .c file is compiled.)

If (2) is not correct, then (1) is not needed either. The old .d files from the last compilation pass are sufficient to know which files need to be recompiled in the current compilation pass. Make does not need to know the dependencies of the .d files themselves, it just needs to load all the existing .d files at startup.

EDIT: Yep, I'm fairly confident this works :D. I don't know if whoever wrote that manual page knew about -MD, but I think it might be newer than -M, which would explain it.

Re: A Tutorial on Portable Makefiles

#66

Post by plorkyeran »

throwaway613834 wrote: post #40
albinofrenchy wrote: post #39

Earlier quoted context omitted.

AFAIK cmake works perfectly fine with relative paths; what breaks?

I don't think it works: https://ofekshilon.com/2016/08/30/cmake-rants/

The author of that blog post wants something fundamentally different from what CMake is. The project files generated by it are just temporary build intermediates, and it makes no more sense to distribute them than it would the .d files generated by gcc -MD. Perhaps he is expecting it to be like autotools, where the generated Makefiles can be bundled in your tarballs so that people do not need autotools installed to build your project, but no meta-build-system other than autotools that I have encountered even tries to support that.

The author's claim that "a CMake-generated project cannot possibly be the final say" is so incorrect that I am not surprising that he found CMake frustrating. It is quite the opposite; if you ever find yourself manually editing the generated project files for any reason other than debugging build-system issues you're doing it wrong and are going to have a very bad time. You must always find a way to express the thing you need within the CMakeFiles.txt, which sometimes unfortunately requires awful hacks.

Amusingly the SO question he links to as evidence that editing the generated project file is required now has an answer saying how to do it properly...

Re: A Tutorial on Portable Makefiles

#67

Post by chubot »

rossy wrote: post #65
chubot wrote: post #62

Earlier quoted context omitted.

I just tried this, and it seems wrong on two counts: 1) The sed magic is required for adding the dependencies of the .d file itself. For example, if the timestamp of your header changes, it may have acquired new dependencies, and a new .d file has to be generated BEFORE make determines if the .c file is out of date. See: https://www.gnu.org/software/make/manual/html_node/Automatic... The purpose of the sed command is…

(2) is not quite correct. The old .d file from the previous compilation is actually all you need to determine whether the .c file needs to be recompiled. It works in all cases. If the .c file is new (or you're doing a clean rebuild of the whole project,) it will always be compiled, because there will be no corresponding .o. If the .c file, or any of the .h files in the old .d gain new header dependencies, they must h…

Hm yes I just figured that out the hard way -- .

This feels hacky, but yes it seems to work. I'll think about it a bit more. (I might clone this feature for a build tool -- since the gcc/Clang support is already there, it seems like any serious build tool needs this. Although some have their own #include scanners which is odd.)

Thanks for the information!

Re: A Tutorial on Portable Makefiles

#68

Post by plorkyeran »

rossy wrote: post #65
chubot wrote: post #62

Earlier quoted context omitted.

I just tried this, and it seems wrong on two counts: 1) The sed magic is required for adding the dependencies of the .d file itself. For example, if the timestamp of your header changes, it may have acquired new dependencies, and a new .d file has to be generated BEFORE make determines if the .c file is out of date. See: https://www.gnu.org/software/make/manual/html_node/Automatic... The purpose of the sed command is…

(2) is not quite correct. The old .d file from the previous compilation is actually all you need to determine whether the .c file needs to be recompiled. It works in all cases. If the .c file is new (or you're doing a clean rebuild of the whole project,) it will always be compiled, because there will be no corresponding .o. If the .c file, or any of the .h files in the old .d gain new header dependencies, they must h…

The problematic case is with generated header files. Suppose foo.c includes foo.h, where foo.h is generated by a separate command. On a clean build, there's nothing telling Make that it needs to build foo.h before foo.c, so it may not happen (and worse, it may usually happen but sometimes not when doing parallel builds). A separate invocation of `gcc -MM` works for this, as when it generates the dependency information for foo.c it will see that it needs foo.h before you do the actual build.

Personally I've never found it too burdensome to just manually specify dependencies on generated files.

Re: A Tutorial on Portable Makefiles

#69

Post by throwaway613834 »

plorkyeran wrote: post #66
throwaway613834 wrote: post #40

Earlier quoted context omitted.

I don't think it works: https://ofekshilon.com/2016/08/30/cmake-rants/

The author of that blog post wants something fundamentally different from what CMake is. The project files generated by it are just temporary build intermediates, and it makes no more sense to distribute them than it would the .d files generated by gcc -MD. Perhaps he is expecting it to be like autotools, where the generated Makefiles can be bundled in your tarballs so that people do not need autotools installed to b…

> The project files generated by it are just temporary build intermediates, and it makes no more sense to distribute them than it would the .d files generated by gcc -MD

Nobody was talking about redistributing anything. I just said I should be able to move the folder from folder A to folder B.

Not sure what to even reply to the rest of your comment...

Re: A Tutorial on Portable Makefiles

#70

Post by plorkyeran »

flukus wrote: post #45
cmm wrote: post #37

where, except for Windows, is requiring GNU Make a problem?

I've heard BSD and mac both use much older, non-GPLv3 versions of make. Portability means supporting older versions too.

/usr/bin/make on macOS is GNU Make 3.81, which is fairly ancient (2006) but still a much more featureful target than POSIX make.

Installing GNU make on BSDs is not difficult; for any project with a nonzero number of dependencies it's likely to be the most trivial one. If you're writing a C89-zero-non-posix-dependencies library then sure, you can't use GNU make, but you also probably only have a ten line makefile anyway.

Post reply on HN