Live data from Hacker News

Autoconf makes me think we stopped evolving too soon

rachelbythebay.com

21–30 of 169 posts

Re: Autoconf makes me think we stopped evolving too soon

#21

Portability. A test to get the answer to "what the fuck am I running on and what does it support" is more portable and robust than thousands of "flavours" manually configured in /etc/whatamieven.conf The author misses that the buildtime magic for the xz exploit is not in the m4 file but in an obfuscated, compressed, encrypted, binary disguised as a test file that alters the build process at multiple stages (configure…

From the article:

That means it's totally normal to ship all kinds of really crazy looking stuff, and so when someone noticed that and decided to use that as their mechanism for extracting some badness from a so-called "test file" that was actually laden with their binary code, is it so surprising that it happened? To me, it seems inevitable.

Yeah, no. The author is well aware of how and why autotools are not awesome but also with the background of why they exist.

A better argument can be made that the act of compiling a binary / obfuscating / minifying code instead of interpreting code directly is the fault.

I can’t decide if you’ve never worked in systems software or trying to be hyperbolic. Given that it’s HN I’ll assume the best. But who do you think would do the interpreting? The priests at Delphi?

Re: Autoconf makes me think we stopped evolving too soon

#22

Portability. A test to get the answer to "what the fuck am I running on and what does it support" is more portable and robust than thousands of "flavours" manually configured in /etc/whatamieven.conf The author misses that the buildtime magic for the xz exploit is not in the m4 file but in an obfuscated, compressed, encrypted, binary disguised as a test file that alters the build process at multiple stages (configure…

[deleted]

Re: Autoconf makes me think we stopped evolving too soon

#23
post #4

You can pre-answer most autoconf sections using /etc/config.site, and a system distributor can create an /usr/share/config.site[1]. It's just really hard to get right, nobody uses it effectively. Autoconf also has a config.cache[2], which it uses to store answers to reuse across multiple of the same checks and reruns of the configure script. [1]: https://www.gnu.org/software/autoconf/manual/autoconf-2.63/h... [2]: ht…

I use autotools for my C++ projects and I hate it (in my weak defense, I hate cmake more). config.cache helps (and ccache helps a lot, too; most of my ccache hits are on stupid autoconf test programs; ya gotta use ccache if you use autoconf), but rerunning configure is still s l o w. Worse, automake and autoconf have a cyclic dependency, so if you're a developer (as opposed to a user building from source on a system)…

libtool does serve a purpose: it causes plain 'make' to produce shell script wrappers as outputs, which is maybe kinda sorta vaguely useful when developing and utterly and completely obnoxious if you want an actual ELF file.

So you end up running 'make install' when you didn’t actually want to install or trying to remember how to find the actual ELF files. And you then wonder why 'make install' appears to be building things instead of just installing them, and you curse at libtool.

In its slight defense, the magic incantation to convince gcc (or, yikes, pre-gcc cc or ld or…) to competently produce a DSO is both nontrivial and platform dependent. Last time I tried to get all the options right off the top of my head, sensible defaults were not a thing.

Re: Autoconf makes me think we stopped evolving too soon

#24
post #4

You can pre-answer most autoconf sections using /etc/config.site, and a system distributor can create an /usr/share/config.site[1]. It's just really hard to get right, nobody uses it effectively. Autoconf also has a config.cache[2], which it uses to store answers to reuse across multiple of the same checks and reruns of the configure script. [1]: https://www.gnu.org/software/autoconf/manual/autoconf-2.63/h... [2]: ht…

I use autotools for my C++ projects and I hate it (in my weak defense, I hate cmake more). config.cache helps (and ccache helps a lot, too; most of my ccache hits are on stupid autoconf test programs; ya gotta use ccache if you use autoconf), but rerunning configure is still s l o w. Worse, automake and autoconf have a cyclic dependency, so if you're a developer (as opposed to a user building from source on a system)…

I don't know how you can think CMake is worse than autoconf. At least it has a declarative approach with good mechanisms for reusing common logic. Its much easier for IDEs/editors to work with it.

Re: Autoconf makes me think we stopped evolving too soon

#26
post #15
post #7

A better solution is just to write a plain ass shell script that tests if various C snippets compile. https://github.com/oilshell/oil/blob/master/configure https://github.com/oilshell/oil/blob/master/build/detect-pwe... Not an unholy mix of m4, shell, and C, all in the same file. --- These are the same style as a the configure scripts that Fabrice Bellard wrote for tcc and QEMU. They are plain ass shell scripts, beca…

> A better solution is just to write a plain ass shell script that tests if various C snippets compile. Then standardize those scripts and share them so others can benefit from code reuse. Then build a configuration tool to make it easy for everyone to define collections of tests applicable to their software. Then create a meta-level scripting system so all of the above can be integrated into the software build proce…

If you redid all that work, and managed to avoid m4, it would still be a win!

I believe m4 was used because some Unix shells at the time did not have functions. All shells have functions now -- it's in POSIX.

If you have shell functions, then you can have modularity without concatenating blocks of code. You can make a library of functions, and then allow people to reuse them.

It could just be

    source lib_configure.sh
    
    detect_foo
    detect_bar
etc.

POSIX shell can do everything that m4 can, and it's MORE deployed, not less. (Related: some people missed that m4 never runs on the end user machine which compiles the source code -- only on the machine that makes the tarball from the repo.)

Re: Autoconf makes me think we stopped evolving too soon

#27
post #7

A better solution is just to write a plain ass shell script that tests if various C snippets compile. https://github.com/oilshell/oil/blob/master/configure https://github.com/oilshell/oil/blob/master/build/detect-pwe... Not an unholy mix of m4, shell, and C, all in the same file. --- These are the same style as a the configure scripts that Fabrice Bellard wrote for tcc and QEMU. They are plain ass shell scripts, beca…

eww. “Try to compile and see if it fails” is awful. I have never and would never do that. That’s so gross.

It's literally the only way to be absolutely certain a specific feature is available. Version checks, shibboleth strings like __GLIBC__, documentation, all of these things can be subverted or lied to. In the end, verifying the desired behavior is present is the only correct move.

Re: Autoconf makes me think we stopped evolving too soon

#28
post #13

Earlier quoted context omitted.

I use autotools for my C++ projects and I hate it (in my weak defense, I hate cmake more). config.cache helps (and ccache helps a lot, too; most of my ccache hits are on stupid autoconf test programs; ya gotta use ccache if you use autoconf), but rerunning configure is still s l o w. Worse, automake and autoconf have a cyclic dependency, so if you're a developer (as opposed to a user building from source on a system)…

I'm be written lots of C code that runs on lots of architectures & platforms: PDPs, x86, ARM; Windows, BSD, Linux, VMS. My projects have spanned them gamut from tiny to gargantuan (millions of lines of code). I've never needed autoconf... what's it even for !?

In decades past, if you were developing software for Unix you would inevitably run across compatibility problems. Even with the POSIX standard, every Unix was trying new things, inventing new things, or just implementing the same thing in slightly different ways. So your programs had to have a bunch of macros to switch between different variations. Some API would have two arguments on Unix A but three on Unix B, so you would have to turn that api call into a macro that would insert the missing third argument or whatever. But now your users needed to know which configuration of your software to use. You could try documenting all the little knobs and switches and make them choose the correct values, but someone hit on a clever idea.

Just write a little program that tries calling the API with two arguments, and see if it compiles. If it does, you could automatically define THAT_API_HAS_TWO_ARGS and your code will do the right thing. If it fails to compile, then define something else.

So you start writing a shell script that will run through all of these tests and dump out a header containing all of these configuration choices. Call the script `configure` and the header `config.h` and you are good.

Except that actually writing a shell script that is really actually POSIX correct is pretty hard. And the code is pretty repetitive. Mistakes start to creep in, bashism break things when it’s actually run under `sh` and so on. And it turns out that all the little tools we rely on, like find and grep and sed and awk, are all different on different platforms too! Turns out that nice option you were passing to grep is just an extension, only present on your specific flavor of unix.

So the next clever idea was to write a program that generates the configure script. The repetitive bits can mostly be solved by text substitution, so someone had the bright idea to use M4. Thus autoconf was born.

Autoconf solved a lot of problems. It ensured that the configure script was actually POSIX correct. It did this even though the generated script was even more complex than ever, with extra logging and tracing and debug output that most people never look at, but which is actually super useful for debugging. The configure script you had written by hand didn’t have any of that!

And autoconf came with baked–in knowledge about all the things you might need to test for. Any time someone discovered some new difference between Unices, new tests were added to autoconf for it. Autoconf users hardly had to do any more work than to look up the name of the macro that it would define for you.

Of course, M4 causes actual brain damage in regular practitioners, and nobody cares to write software for any Unix except Linux these days. Even the BSDs are an afterthought. Technically OpenSolaris is still out there, with superior features like ZFS and Zones and whatnot, but that’s a lot of extra work. So autoconf is a dinosaur, solving problems that nobody really has any more.

Rachel is not wrong that it’s crazy to rerun the same configure script over and over, but really only developers have to do that. Most people just run the thing once when they install your software, and never see it again. And most people don't even do that any more, because they download binaries from their package manager most of the time. It’s just the distro packagers that actually run your configure script.

And she’s right that the answers configure gets should be cached system–wide. Autoconf does support that, but it’s harder to do correctly than she remembers. And by the time it was possible usage was already waning in practice.

Re: Autoconf makes me think we stopped evolving too soon

#29
...But I like seeing hundreds of lines of "yes"es and success messages fly by at lightning speed before the inevitable compilation failure for the first line of actually incompatible (and non-trivial-to-fix) code. It makes me feel like I actually have a shot at compiling liboldafgnuglibxx3 from 2004 and that my efforts aren't futile.

Re: Autoconf makes me think we stopped evolving too soon

#30

Is anyone doing and publicizing the git clone vs. tarball work in other projects? I suppose if something has been found, it might be too soon to disclose publicly.

Note that this sort of comparison isn't necessarily trivial. I have code where running `foo --version` out of the git repository will print "foo @@VERSION@@" because that gets substituted with the actual version number as part of the release process, for example.
Post reply on HN