Live data from Hacker News

Autoconf makes me think we stopped evolving too soon

rachelbythebay.com

131–140 of 169 posts

Re: Autoconf makes me think we stopped evolving too soon

#131
post #52

Earlier quoted context omitted.

Yup, I call that the Perlis-Thompson Principle -- because Ken Thompson made a similar combinatorial argument about software composition: you should design it around "one thing". https://www.oilshell.org/blog/2021/07/blog-backlog-1.html#co... Files had structure on pre-Unix OSes, but they don't on Unix, because it doesn't compose. The "Uniform Interface Constraint" of REST is the same thing, and resembles a file syste…

> What about a Clojure program and a Common Lisp program? Probably a byte stream. (Ironically, S-expressions have no commonly used "exterior" interchange format) Probably because the details of S-expressions differ underneath the hood - a simple (foo ()) can mean a different thing on CL (where the second element is the symbol NIL) than on Scheme (where it is an empty list and not a symbol); also because Clojure intro…

Yeah the way I frame it is that Lisp is about "interior" composition (functions and data), not "exterior" (processes andf iles)

Narrow Waists Can Be Interior or Exterior: PyObject vs. Unix Files

https://www.oilshell.org/blog/2023/06/narrow-waist.html

However the "exterior" extension seems obvious to me -- why hasn't anyone produced a distributed Lisp?

Well I guess Clojure/EDN is that, but nobody has produced a POLYGLOT system involving distributed Lisp. Common Lisp users probably don't use EDN very much, I gather.

---

I found this POSE exterior s-expression format, but it seems to have little interest from the Lisp community:

https://lobste.rs/s/lwf4jv/pose_portable_s_expressions_pose_...

It seems like JSON will have an "impedance mismatch" with Lisp data structures, but yeah that's the cost of interoperability

Re: Autoconf makes me think we stopped evolving too soon

#132
post #9
post #4

You can pre-answer most autoconf sections using /etc/config.site, and a system distributor can create an /usr/share/config.site[1]. It's just really hard to get right, nobody uses it effectively. Autoconf also has a config.cache[2], which it uses to store answers to reuse across multiple of the same checks and reruns of the configure script. [1]: https://www.gnu.org/software/autoconf/manual/autoconf-2.63/h... [2]: ht…

In my experience, config.cache fails to properly separate platform feature tests from build configuration options, so you need to delete config.cache if any build options change. It is extremely brittle and buggy. config.cache is definitely not suitable as a basis for for platforms to provide pre-cooked answers to platform feature tests.

In other words, it's not just as simple as "figure out how to compile things on this system, once, and then reuse that information in the future". Different projects can be compiled with different options, cross-compilation is a thing, etc. As soon as you try to solve this with a simple global `site_config.h` you discover that that doesn't quite work.

Re: Autoconf makes me think we stopped evolving too soon

#133
post #97
post #83

Earlier quoted context omitted.

IMO Cmake's "killer app" is that it will generate projects for Visual Studio and Xcode, which allows you to get access to the Windows/Mac specific tooling that those products offer. It's also a LOT faster on Windows than Autotools because of how NTFS performs poorly when dealing with writing a ton of small files. I agree that its syntax is ugly but once I got beyond that I found it fairly simple to work with. The mos…

Rather than the "killer app", that's the only thing that cmake does better than other systems, and the only reason for anybody to use cmake.

It's kind of weird that these days, many (most?) projects use Cmake directly as their build tool. They don't generate a VC or Xcode project and then work from that; they generate a VC project and then immediately build it. The CMakeLists.txt effectively is the project definition. VS now has support for opening CMake projects directly, without creating a VS project file from them. So the thing that made CMake useful (and what also makes it complicated...), is now used less and less.

Re: Autoconf makes me think we stopped evolving too soon

#134
One of the worst things about Autoconf is the way it is deployed in the GNU Project's own programs.

Typically, GNU project programs assume that anyone working on the program (cloning the repo, looking for bugs, making changes, ...) has the Autotools installed. Not just any Autotools but the exact version the program wants.

Only the "release tarball" if a GNU program contains the generated ./configure script that can run on a machine that doesn't have Autotools installed. You have to do some "make boostrap".

So to make a source code release, they can't just tag a code baseline and serve that; they have to generate the release source!

See the stupidity? The source code that is being released has to be compiled from the real source source.

Now this makes life hell if you have to, say, git bisect to find a commit that introduces a problem. To be sure you're building each bisect step correctly, you have to disclean everything and re-run the autoconf bootstrapping, then ./configure and so on. The old code you're checking out may call for different Autotools versions from the trunk head.

Working with some GNU programs is basically hell.

Re: Autoconf makes me think we stopped evolving too soon

#135

Is anyone doing and publicizing the git clone vs. tarball work in other projects? I suppose if something has been found, it might be too soon to disclose publicly.

Note that this sort of comparison isn't necessarily trivial. I have code where running `foo --version` out of the git repository will print "foo @@VERSION@@" because that gets substituted with the actual version number as part of the release process, for example.

A first step could be performance benchmarking a tarball release vs something built from a git clone, maybe? Just a guess since that would be similar to how this was found.

Re: Autoconf makes me think we stopped evolving too soon

#136
post #52

Earlier quoted context omitted.

Yup, I call that the Perlis-Thompson Principle -- because Ken Thompson made a similar combinatorial argument about software composition: you should design it around "one thing". https://www.oilshell.org/blog/2021/07/blog-backlog-1.html#co... Files had structure on pre-Unix OSes, but they don't on Unix, because it doesn't compose. The "Uniform Interface Constraint" of REST is the same thing, and resembles a file syste…

What benefit is this? I mean, I love the look of it, but how can it help me from day dot?

There are many answers, but one situation is

    1. you already have a bash script (e.g. in your CI)
    2. you want to use an array or dict
Bash arrays and dicts have all sorts of warts. You have to remember to use "${array[@]}" (exactly like that - https://www.oilshell.org/blog/2016/11/06.html )

You can upgrade to YSH, and start using our Python/JS like arrays and dicts instead. And then you can write them to JSON and upload them somewhere.

---

Another situation is if you use ShellCheck, but you want even more strictness at RUNTIME, then you can do

    shopt --set strict:all
Oils is very ambitious -- a complete overhaul of shell -- but that's a couple immediate things.

https://www.oilshell.org/blog/2021/01/why-a-new-shell.html

Re: Autoconf makes me think we stopped evolving too soon

#137
post #36

Earlier quoted context omitted.

Alan Perlis Epigram #9: "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures." Unix settled for string as a data structure and it's little too low level. All Lisp programmers know that s-expressions would have been the right choice. JSON works also.

IMO a big reason Python is popular is because so many big seemingly unrelated librarues (AI things, scipy, Pillow, pandas, GDAL rasters, matplotlib, etc) all work with the "Numpy array" datatype and can therefore do things with each other's data.

Yup, I agree that Python composes like Unix does, which is a reason for its phenomenal growth:

Narrow Waists Can Be Interior or Exterior: PyObject vs. Unix Files - https://www.oilshell.org/blog/2023/06/narrow-waist.html

The appendix links to: Why is Python the language of machine learning?

https://news.ycombinator.com/item?id=29171519

In particular, I highly recommend this bit of history from NumPy creator Travis Oliphant:

https://www.youtube.com/watch?v=gFEE3w7F0ww

Re: Autoconf makes me think we stopped evolving too soon

#138
> So, okay, fine, at some point it made sense to run programs to empirically determine what was supported on a given system.

That stopped making sense several decades ago, when FOSS began to be heavily used for embedded systems, where you are crosscompiling. Autoconf supports cross-compiling, but very badly. Programs not prepared with crosscompiling in mind will do stupid things at configure time that require a compiled program to be executed. For that reason, distros ended up using Qemu for cross builds.

> What I don't understand is why we kept running those stupid little shell snippets and little bits of C code over and over. It's like, okay, we established that this particular system does with two args, not three. So why the hell are we constantly testing for it over and over?

Autoconf does have a cache system for this. I think it's only used within one package, not globally. There may be some knobs and levers to share autom4te.cache stuff between projects.

Even when the configuration is not cached for the purposes of the test itself, obviously the information is cached in the generated config.h and config.make type files; once the program is configured, each time you recompile it, it's not testing for whether foobar has two args, not three; it's just referring to HAVE_TWO_ARG_FOO or whatever.

There could be a global cache for autoconf tests, or at least the stanadard ones built into autoconf. Each test would have to have a unique ID, which changes whenever it changes between autoconf versions.

The cache would have to distinguish targets. Packages building for the build machine itself must not share configuration results with packages built for the target.

Even if it were done right in these regards, there is a risk of introducing bugs due to package build interaction through such a shared cache. A clean build of any package is not really clean if it refers to material from a previous build, possibly of another package. I can see distro builds opting out of something like this, and they would be the primary audience, since it is distro builds that are hurt by hundreds of packages executing essentially the same tests over again.

Re: Autoconf makes me think we stopped evolving too soon

#139
post #131

Earlier quoted context omitted.

> What about a Clojure program and a Common Lisp program? Probably a byte stream. (Ironically, S-expressions have no commonly used "exterior" interchange format) Probably because the details of S-expressions differ underneath the hood - a simple (foo ()) can mean a different thing on CL (where the second element is the symbol NIL) than on Scheme (where it is an empty list and not a symbol); also because Clojure intro…

Yeah the way I frame it is that Lisp is about "interior" composition (functions and data), not "exterior" (processes andf iles) Narrow Waists Can Be Interior or Exterior: PyObject vs. Unix Files https://www.oilshell.org/blog/2023/06/narrow-waist.html However the "exterior" extension seems obvious to me -- why hasn't anyone produced a distributed Lisp? Well I guess Clojure/EDN is that, but nobody has produced a POLYGL…

> However the "exterior" extension seems obvious to me -- why hasn't anyone produced a distributed Lisp?

How about Lisp Flavored Erlang?

Re: Autoconf makes me think we stopped evolving too soon

#140
post #48

Earlier quoted context omitted.

Congrats on the clean living, but it remains the norm regardless. Both CMake and Autotools use this approach. C and C++ aren't the only languages for which this is true, by the way. It is de rigueur these days to have versioned releases for the purposes of feature gating, but there's half a century of code out there written in languages without that, and explicit feature testing will be with us as long as that code i…

> it remains the norm regardless. Both CMake and Autotools use this approach Yes, it is the norm (in certain circles, but not others). No, it doesn't have to be that way. CMake and Autotools are both genuinely awful. The world would be a better place if Bazel/Buck2/similar were the norm. I find it extraordinarily bizarre how defensive Linux people get when I say "the status quo is actually bad, but it doesn't have to…

I didn't say Autotools and CMake weren't awful, I said that approach exists for a reason. You seem not to understand; maybe it's my poor communication skills. And I don't appreciate being called "Linux people," because I am not one, thanks. Not sure why personal attacks are in play here.

The thing you're missing is that Bazel, Buck2, Zig, and all these things that do not require the compile-testing approach is they are not portable. They support Windows, MacOS, and Linux, by special-casing each platform, and even the Linux support is restricted to a few mainstream distros. This is a perfectly valid technical decision, and one I support, but it was not possible for decades.

Now people take Linux for granted, but until relatively recently you had dozens of almost-compatible Unix clones from dozens of vendors, of which Linux was but one. All of them had to be, essentially, written from scratch or based on a BSD release, with the differentiating features implemented on top by a given vendor. Since different developers had different interpretations and priorities for "Unix compatibility," this led to a combinatorial explosion of possible software configurations even just at the libc level, much less kernel API. The Single Unix Specification and POSIX standards were assembled to try to correct this, but there was no way to go back and un-differentiate all these Unix clones. So developers targeted SUS or POSIX, and mostly got what they needed, but something had to account for all the little differences that remained. This was the purpose of Autotools, and Cmake was written because Autotools sucked.

So, these things you're praising which avoid that whole scene, are able to do so because, for better or worse, Linux won. The other Unix clones have either completely died out (SCO Unix etc), are dying out quickly (Solaris etc), or have stabilized into their own distinct platforms (FreeBSD, MacOS, etc).

It has nothing to do with the build system. It has everything to do with platform consistency. That's why Plan 9 can do it without any build system beyond mk (a weird clone of Make), and why the FAANG companies can crank out build systems as get-me-promoted projects -- they're targeting three stable operating systems, instead of an uncountable number of weird almost-compatible monsters.

But again, from this terrible Unix scenery emerged a remarkably robust and portable set of tools (gcc, the gnu coreutils, etc) which became the de facto standard across all the Unix clones. People liked them because they worked the same on every Unix, and because (thanks to Autotools) they could build on every Unix. Because they had this versatile build system, when Linux came around the GNU tools were the easiest to port. Because they were the main tools running on Linux, and people had experience with them from other Unix clones, Linux adoption was quick and easy. And so it grew in popularity...

So yes, I mean "because of." It's easy to hate on complicated old rickety shit like Autotools, and I will be happy when I never have to deal with it again, but pretending it has no value and typing in "ewww" is needlessly dismissive and denies the actual importance this code had in getting us to a place where we can live without it.

Post reply on HN