Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autotools Mythbuster

autotools.io

Post reply on HN
Print view View original

51–60 of 88 posts

Re: Autotools Mythbuster

#51

Post by jerf »

DasIch wrote: post #3

Learning about autotools is an amazing source of fremdscham. The whole compilation/packaging stuff is hard to get right but autotools looks like someone completely gave up on producing a good, decent or even bad solution and just tried whatever didn't completely and utterly fail at the task. That everyone else just kind of seemed to go with it... there are just no words to describe how embarrassing that looks.

A solution can be no simpler than the essential complexity of the problem it is solving. It is not clear to me that the essential complexity of the problem is significantly simpler than the autotools solution. Across the totality of all C/C++-using systems, across all of the mostly-POSIX and vaguely-POSIX-ish systems, across all the libraries, the toolkits, the OSes, across the decades, across all of what it is trying to span, the essential complexity is quite large. Contemplating all that probably makes autotools come out looking pretty good in context.

I'd observe that every major modern language since then has taken an approach that in one way or another, obviates the need for a full autoconf. C's contribution to large-scale code reuse was largely showing all the ways in which it could go wrong, and encouraging future language developers to get it more right to start with, which they mostly have.

(Note I'm not saying there aren't any problems with the way any modern language works. But they're still all simpler, generally more powerful, and even in their worst case, nothing like as bad as the autoconf-world's worst case.)

Re: Autotools Mythbuster

#52

Post by MichaelMoser123 »

david-given wrote: post #42
MichaelMoser123 wrote: post #25

GNU make has macros too, but these were added much later than 1991; Interesting if make macros were available earlier, would people have used them instead of autotools ? Thank you for the link on autotools, i will try to apply it if i am ever forced to do something with autoconf/autotools.

Trying to do anything non-trivial in GNU make macros is the most bewilderingly awful programming experience it is possible to have. They are horribly broken in every respect, from little stuff like awful syntax (whitespace is insignificant except where it's significant!) and poor error checking (detecting undefined variables? Who cares?) to the really big stuff (can't do arithmetic, despite having core functions whic…

I don't know, Make is difficult because we are used to think in procedures and not in rules; and the rule and macro syntax is a bit weird; it takes time to master it.

i have a makefile/make system that uses gnu make macros; (here http://mosermichael.github.io/cstuff/all/projects/2011/06/17.... ) this saves you from repeating the same make constructs many times over, in the following example you do a static library and executable.

  1: TOPDIR=../..
  2:
  3: # - declare build targets. (built with make)
  4: TARGETS:=shlib slibuser
  5: 
  6: # - slib target is a static library -
  7: shlib_TYPE=lib
  8: shlib_SRC=slib.c
  9:
  10:
  11: # - slibuser target is a executable using slib -
  12: slibuser_TYPE=exe
  13: slibuser_SRC=slibuser.c slibuser2.c slibuser3.c
  14: slibuser_LIBS=shlib
  15:
  16: include $(TOPDIR)/rules.make
At a previous job they had an even more convoluted make system - it was simulating macros: the makefile was including another generic make file; this included make file was writing hidden files that contained the make rules ; then as the last step these generated files were included.

Re: Autotools Mythbuster

#53

Post by nickpsecurity »

jerf wrote: post #51
DasIch wrote: post #3

Learning about autotools is an amazing source of fremdscham. The whole compilation/packaging stuff is hard to get right but autotools looks like someone completely gave up on producing a good, decent or even bad solution and just tried whatever didn't completely and utterly fail at the task. That everyone else just kind of seemed to go with it... there are just no words to describe how embarrassing that looks.

A solution can be no simpler than the essential complexity of the problem it is solving. It is not clear to me that the essential complexity of the problem is significantly simpler than the autotools solution. Across the totality of all C/C++-using systems, across all of the mostly-POSIX and vaguely-POSIX-ish systems, across all the libraries, the toolkits, the OSes, across the decades, across all of what it is tryin…

Even as an autotools critic, I totally understand that argument about the steps that were necessary for dealing with those portability problems. Well, way back in the day when they existed to the degree they did. Today, there should be much less complexity as the systems are more alike than different. What complexity is there is mostly a relic from days gone by that never got cleaned up because nobody wants to invest the effort.

Re: Autotools Mythbuster

#54

Post by the_why_of_y »

david-given wrote: post #50
scrollaway wrote: post #47

Earlier quoted context omitted.

Fun story, I have a set of scripts that extract, decompile and process various sets of files, outputting them into a build directory. At first I wrote shell scripts to automatically do all this, then I figured this is the sort of thing Make was built for (right?). This is what I ended up with after days of work: https://github.com/HearthSim/extract-scripts/blob/master/Mak... Turns out, Make kind of just assumes you'r…

One thing that make appears completely unable to do cleanly is to build things twice, with different flags. You know, like 'build a release version and a debug version'. I've ended up having to write self-writing makefiles to do that. It's not as nasty as it sounds (but still pretty nasty): https://github.com/davidgiven/wordgrinder/blob/master/Makefi... Plus, 'set a flag for just this file' is another thing that's ap…

If you write make files to build everything in twice, release and debug, you're doing it wrong. The proper way to do this is to write make files so you can do "out-of-tree" builds, i.e. your build directory is outside your source directory and each build directory has its own configuration.

This has the significant advantage that you can not just build 2 or 3 or however many different configurations you would think important enough to hard code in the build system from one source tree, but as many as anyone who builds it happens to need.

Incidentally this is one thing that GNU autotools get right.

It is true that if you use target specific variables you have to ensure yourself that you set them in such a way that no non-deterministic behavior can occur; it would be great if make reported a warning in such cases. Another problem with them is that you can't invoke make with a single object file target any more if that requires a variable set on and inherited from the corresponding link target to compile.

Re: Autotools Mythbuster

#55

Post by kazinator »

jessaustin wrote: post #44
kazinator wrote: post #30

Earlier quoted context omitted.

I recommend a hand-crafted configure script, which provides the same usage interface to package builders, without Autocrap: E.g.: http://www.kylheku.com/cgit/txr/tree/configure

That script is 2500 lines long, and appears to have been written by a wizard. If my only other option was producing something like that, I'd just bite the bullet and use autotools. (IDK, that may be your point?)

Do you honestly think that I gradually developed a 2500 line configure script by hand over more than six years for an important project of mine ironically, in order to promote Autocrap?

The script detects just what is necessary for the program and is easy for me to understand, extend, port and debug.

You can fix it right on a given target platform where you're trying to get it working.

You can't do that with an Autoconf configure script (in a way that you can merge back to your stream), because it's generated. You need an installation of Autoconf.

If you want to patch the Autoconf-generated configure script with minimal changes, just reflecting the topic of the change you're making, you need the exact same version of Autoconf with which it was generated, otherwise there will be irrelevant diffs all over the script.

(I learned this when I developed an embedded, cross-compiled LInux distro from scratch. On my development system, I had about half a dozen installations of different versions of Autoconf, so that I could create minimal patches against configure scripts!!!)

And, of course, you first have to work backwards to figure out how to coax the script behavior you want indirectly through the m4 macro files. Forget about "I just want to make this tiny tweak in one command and it will Just Work".

In my script, you just ... open the script and write the necessary code. Then regression-test it on the other targets, do any necessary tweaking and commit.

Once you have written one of these scripts, you can easily re-use it. My above script is BSD-licensed. Anyone can take it and cut out whatever they don't need. If you want help using it or have any questions, you can fire me an e-mail.

Re: Autotools Mythbuster

#56

Post by dankohn1 »

Could anyone speak to why you would use Autotools today in a new project rather than gyp [0] (created by the Chrome team because Autotools sucks) or waf [1]? How many people really have a requirement to support more than just Windows, OS X, and Linux? I can understand not wanting to rip out a working Autotools implementation of a legacy project, but it's greenfield use should decline to zero.

[0] https://gyp.gsrc.io/ [1] https://github.com/waf-project/waf

Re: Autotools Mythbuster

#57

Post by kazinator »

scrollaway wrote: post #46
adekok wrote: post #36

Earlier quoted context omitted.

Even worse, autotools doesn't have the concept of modularization. Look at the nginx configure scripts. They're hand-written, and modularized. Check for "foo"? There's a shell script. Autotools does the same thing via special-purpose autogenerated shell code. That's why the configure script is 1000's to 10's of 1000's of lines of shell script. It's all auto-generated crap. And autotools doesn't have the concept of boo…

> I've used autotools for decades because everything else is worse. I take it you've used CMake - could you explain in which ways you think it's worse than autotools? I agree it has concerning design flaws, but they don't come anywhere close to autotools' and I'd say CMake is overall dozens of times saner.

Does CMake generate a deliverable which builds on a machine that doesn't have CMake installed? With full incremental build support and all, so reasonable development can take place on that system?

Re: Autotools Mythbuster

#58

Post by kazinator »

dankohn1 wrote: post #56

Could anyone speak to why you would use Autotools today in a new project rather than gyp [0] (created by the Chrome team because Autotools sucks) or waf [1]? How many people really have a requirement to support more than just Windows, OS X, and Linux? I can understand not wanting to rip out a working Autotools implementation of a legacy project, but it's greenfield use should decline to zero. [0] https://gyp.gsrc.io/…

autotools isn't mainly a build system; it's a system for preparing a program which will self-configure and build in various systems. Those systems themselves do not have any special tools installed. No Autotools, no GNU-anything.

Autotools was intended to solve the problem of how to give the same tarball of code to a user of SunOS 4, Ultrix, HP-UX 8, and so on, such that each of those users could just unpack the tarball, run ./configure and then make. (Using the make that came with their OS, not CMake, not GNU Make, not gyp and so on).

Some of the requirements don't hold as much any more; you can rely on newer POSIX shell features in a configure script, and GNU make is more widely available and so on, but the basic idea is solid: no requirement for any exotic build system just to get the program running.

It's the implementation that sucks in Autotools. The high level ideas are solid.

Re: Autotools Mythbuster

#59

Post by kazinator »

nickpsecurity wrote: post #53
jerf wrote: post #51

Earlier quoted context omitted.

A solution can be no simpler than the essential complexity of the problem it is solving. It is not clear to me that the essential complexity of the problem is significantly simpler than the autotools solution. Across the totality of all C/C++-using systems, across all of the mostly-POSIX and vaguely-POSIX-ish systems, across all the libraries, the toolkits, the OSes, across the decades, across all of what it is tryin…

Even as an autotools critic, I totally understand that argument about the steps that were necessary for dealing with those portability problems. Well, way back in the day when they existed to the degree they did. Today, there should be much less complexity as the systems are more alike than different. What complexity is there is mostly a relic from days gone by that never got cleaned up because nobody wants to invest…

As someone who maintains a hand-written configure script for a project that I build on just a handful of platforms, I have to disagree. There is plenty of cruft there to detect and take care of.

For example, recently I discovered that -D_FILE_OFFSET_BITS=64 works on 32 bit Solaris 10 for large file support. However, fseeko and ftello are not declared. For that you need something special, namely -D_LARGEFILE_SOURCE --- this is not just assumed from _FILE_OFFSET_BITS=64.

New developments in POSIX have created a fuzzy landscape which resembles some of the confusion in 1990 Unixes. This platform needs _POSIX_C_SOURCE=, another has the same thing under _XOPEN_SOURCE. Apple won't declare anything useful to you without -D_DARWIN_SOURCE.

Some platform needed __EXTENSIONS__ to declare struct winsize. I don't remember which one, but my script detects that.

Systems are not becoming more alike. Even when it comes to standard stuff, they vary in their conformance to POSIX, and in how they declare identifiers and the exact headers that have to be included and such.

Re: Autotools Mythbuster

#60

Post by david-given »

the_why_of_y wrote: post #54
david-given wrote: post #50

Earlier quoted context omitted.

One thing that make appears completely unable to do cleanly is to build things twice, with different flags. You know, like 'build a release version and a debug version'. I've ended up having to write self-writing makefiles to do that. It's not as nasty as it sounds (but still pretty nasty): https://github.com/davidgiven/wordgrinder/blob/master/Makefi... Plus, 'set a flag for just this file' is another thing that's ap…

If you write make files to build everything in twice, release and debug, you're doing it wrong. The proper way to do this is to write make files so you can do "out-of-tree" builds, i.e. your build directory is outside your source directory and each build directory has its own configuration. This has the significant advantage that you can not just build 2 or 3 or however many different configurations you would think i…

Sorry, I don't follow. I am doing out-of-tree builds?

The problem is that there is no way to generate multiple files from one rule. If you write:

    $(OBJDIR)/foo.o: src/foo.c
...you get one rule, which will build one object file. If you want to generate two object files with two different settings, you have to have two rules.

    $(OBJDIR1)/foo.o: src/foo.c
    $(OBJDIR2)/foo.o: src/foo.c
There is no way round this. Target-specific variables look like they solve this, but they basically don't work for anything useful. There are only three options here: (1) hardcode multiple rules; (2) use make macros to generate multiple rules; (3) invoke the makefile multiple times with different settings. All of these options are horrible.
Post reply on HN