Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autoconf makes me think we stopped evolving too soon

rachelbythebay.com

Post reply on HN
Print view View original

121–130 of 169 posts

Re: Autoconf makes me think we stopped evolving too soon

#121

Post by adrian_b »

mid-kid wrote: post #113
dima55 wrote: post #95

Earlier quoted context omitted.

Are they though? The "complex .d-file generation" is "gcc -MMD" and a "-include *.d" at the end of the Makefile. You specify the dependencies, and it's on you to get those right, as it is in every other build system. I've never seen a race condition. The problem with Make, is that it works just well-enough to allow people to sorta get it to work without requiring them to learn how to actually use the thing. Then they…

Nice job, you just hit a footgun: try to remove any .h file (and the corresponding #include directive) and see your build just break. You need an empty `%.h: ;` directive for make to ignore missing .h files. Make is full of this stuff and as a distro packager having to deal with each's individual quirks is prone to drive one insane. Please consider using meson, at least.

I do not believe that adding such rules for ignoring files is wise.

I prefer to just run "make clean" after I delete, rename or move source files.

With a properly written "Makefile" nothing else needs to be done and there will never be a broken build. With a properly written "Makefile" nothing has to be done when adding source files.

Re: Autoconf makes me think we stopped evolving too soon

#122

Post by pianoben »

dilyevsky wrote: post #92
pianoben wrote: post #73

Earlier quoted context omitted.

Works great if you follow Google's style of vendoring all of your dependencies. If you try to rely on third-party package managers like Maven or Ivy or whatever C++ uses, you're in for some serious pain. The more you venture into OSS tools like Spring, gRPC, containers, the more you have to incorporate shoddy and poorly-maintained third-party Bazel tools which inevitable stomp on each other in subtle ways. My team ab…

> Works great if you follow Google's style of vendoring all of your dependencies. If you try to rely on third-party package managers like Maven or Ivy or whatever C++ uses, you're in for some serious pain. It sounds like you've mainly dealt with Java because it's not a pain in C/C++ - if you wish you can link dynamically with system libs or vendor .so binaries. And at least for Maven I used rules_jvm_external which w…

I've done some C/C++ with Bazel before. It's nice, except that you end up having to vendor all your dependencies if you want repeatable builds :). Had more success sharing projects and building a community with CMake, horrible as that tool is.

The reason Gradle is a quality-of-life improvement is that we had boxed ourselves into a corner. rules_jvm_external, rules_docker, rules_proto, and a set of others all relied on internal details of other rules (accidentally, because pre-MODULE.bazel everything gets loaded into one giant global namespace and collisions happen). We basically got ourselves locked in, unable to move off of rules_docker, upgrade our JVM beyond 17, or even upgrade Spring, without breaking and having to upgrade everything else.

Gradle, which has first-party support for dependency management, and which is the primary system supported by large OSS vendors (Spring, etc), doesn't have these pitfalls. Poorly-written Gradle plugins can sink you as surely as Bazel packages can, but there is less opportunity for you to discover them.

I'm well aware that we were Doing Bazel Wrong, and I've certainly had my share of pain with Gradle, but never in my life have I seen so much time wasted on a build system. For plain old JVM stuff, Bazel didn't work well for us.

Re: Autoconf makes me think we stopped evolving too soon

#123

Post by fanf2 »

darnir wrote: post #94
db48x wrote: post #28

Earlier quoted context omitted.

In decades past, if you were developing software for Unix you would inevitably run across compatibility problems. Even with the POSIX standard, every Unix was trying new things, inventing new things, or just implementing the same thing in slightly different ways. So your programs had to have a bunch of macros to switch between different variations. Some API would have two arguments on Unix A but three on Unix B, so y…

That's a very nice summary. But you're missing one crucial point. These are GNU tools and if you use them within that ecosystem, you also often use gnulib. Gnulib is effectively the GNU Portability Library. Your fancy, new, auto generated configure script can find all the differences, but someone still needs to account for them and write alternative code to support the various platforms. This is where gnulib comes in…

Yeah, there’s a lot of software that uses autoconf without gnulib. It’s common to include a load of default platform tests in the configure script that are used by gnulib, but without gnulib the software never uses the results of the tests. All these futile tests are an appalling waste of time and energy.

Re: Autoconf makes me think we stopped evolving too soon

#124

Post by [deleted] »

vineyardmike wrote: post #108
Scarblac wrote: post #91

Earlier quoted context omitted.

IMO a big reason Python is popular is because so many big seemingly unrelated librarues (AI things, scipy, Pillow, pandas, GDAL rasters, matplotlib, etc) all work with the "Numpy array" datatype and can therefore do things with each other's data.

Relatedly, this is why big companies (eg Google) all have internal standard interchange formats (eg Protos). It’s so every programmer and every service and every file stored can always be consumed or produced as needed. Even smaller companies I’ve seen end up with an “interfaces” or “types” repo that contains definitions of every external object.

[deleted]

Re: Autoconf makes me think we stopped evolving too soon

#125

Post by kergonath »

ajross wrote: post #47
evmar wrote: post #11

FWIW the sandbox-disabling bit in xz was in CMake logic, not autoconf. (Or at least the obvious one is in the CMake part; maybe the autoconf is separately backdoored.) https://git.tukaani.org/?p=xz.git;a=commitdiff;h=328c52da8a2...

Yeah, this autotools thing is just a chance for people to whine about a favorite hated technology[1]. Clever attackers (and this was a very clever attacker) can hide stuff in anything. Anyone who thinks that CMake or Bazel or Python or whatever can't hide misbehavior is just fooling themselves. [1] And even then it's more hatred of M4 than it is anything in autoconf per se.

> And even then it's more hatred of M4 than it is anything in autoconf per se.

To be fair, M4 is rather…special. And when you add that in a Makefile template with shell thrown into the mix it gets _very_ hairy.

Re: Autoconf makes me think we stopped evolving too soon

#126

Post by nottorp »

forrestthewoods wrote: post #85
marcus_holmes wrote: post #44

Earlier quoted context omitted.

Ahh the eternal cycle of static vs dynamic linking swings around again. "I need common.lib, it gets linked into my application at compile-time" "why does every binary include this common.lib code? Let's pull that out to a shared module so we only have one version of it! See? that saved 35739475749Kb of storage! Everything is smaller and sleeker now!" "which version of common.lib does this application need? Hmm, but t…

> Ahh the eternal cycle of static vs dynamic linking swings around again. Somewhat. Although it's agnostic to static vs dynamic linking. The philosophy goes a bit deeper than that. > See? that saved 35739475749Kb of storage! Storage capacity hasn't been relevant for over a decade. Besides, we foolishly replaced "use shared modules to save storage space" with "build multi-gigabyte docker images because it's the only w…

> Storage capacity hasn't been relevant for over a decade.

> Besides, we foolishly replaced "use shared modules to save storage space" with "build multi-gigabyte docker images because it's the only way to reliably launch a simple program without crashing on startup".

But... you just said it still is :) By two orders of magnitude now.

Re: Autoconf makes me think we stopped evolving too soon

#127

Post by ajross »

kergonath wrote: post #125
ajross wrote: post #47

Earlier quoted context omitted.

Yeah, this autotools thing is just a chance for people to whine about a favorite hated technology[1]. Clever attackers (and this was a very clever attacker) can hide stuff in anything. Anyone who thinks that CMake or Bazel or Python or whatever can't hide misbehavior is just fooling themselves. [1] And even then it's more hatred of M4 than it is anything in autoconf per se.

> And even then it's more hatred of M4 than it is anything in autoconf per se. To be fair, M4 is rather…special. And when you add that in a Makefile template with shell thrown into the mix it gets _very_ hairy.

Oh sure, and that's true enough. Working with autoconf[1] is a giant pain for Kids Today weaned on entirely different idioms[2]. And it's the kind of thing you can't really avoid[3] if you're going to be doing work on these traditional Linux packages. And that sucks, and it makes it look like Linux is a weird kingdom filled with dinosaurs. And that sucks because you still have to work in Linux. So you hate the proximate cause.

But the boring truth is that configuration management is just a hard problem and realistically you'd hate it anyway.

[1] Not so much automake/libtool, which are configured by straightforward idioms and mostly harmless.

[2] Though if you've got some C preprocessor fu, it doesn't look quite so alien.

[3] Though it also needs to be pointed out that 90% of the problem autoconf was originally intended to solve (gratuitously incompatible Unix variants) is basically gone now. You either write to straight ISO-C/POSIX if you're doing simple stuff, work through well-established abstraction libraries with fixed APIs, or you include well-known platform headers (Linux or otherwise) for the complicated bits. No more nonsense testing like "Hm... what header do I need for strncat()?"

Re: Autoconf makes me think we stopped evolving too soon

#128

Post by kergonath »

parasti wrote: post #98

Autoconf has never really made sense to me in the last twenty years or so. At what point does a Makefile-based project start thinking "hmm, we should upgrade to autoconf to solve this"? And what is "this" that they can't solve any other way? Seems like that point never occurs anymore. Either projects start with autoconf or they don't.

To me, the point is when something I write has tor each for BLAS or LAPACK and every single user seems to want it to work with a different library (ATLAS, OpenBLAS, MKL, Accelerate.framework, vanilla BLAS, cuBLAS)… Then, combine that with the various FFT libraries flavours, wherever the fuck hdf5 and NetCDF are, and which goddamned version of MPI they want to use. Not mentioning some computers where they insist on using Intel’s compilers. Or, dog saves us, Cray.

Getting this to work is a major pain, we cannot expect the users to figure it out themselves, and we cannot list all the possible combinations in a Makefile.

We tried several fancy build systems, and every single one breaks in more or less subtle ways.

Re: Autoconf makes me think we stopped evolving too soon

#129

Post by kergonath »

chubot wrote: post #7

A better solution is just to write a plain ass shell script that tests if various C snippets compile. https://github.com/oilshell/oil/blob/master/configure https://github.com/oilshell/oil/blob/master/build/detect-pwe... Not an unholy mix of m4, shell, and C, all in the same file. --- These are the same style as a the configure scripts that Fabrice Bellard wrote for tcc and QEMU. They are plain ass shell scripts, beca…

> OCaml’s configure script is also “normal”

If that’s this OCaml, it has a configure.ac file in the root directory, which looks suspicious for an Autotools-free package: https://github.com/ocaml/ocaml

Re: Autoconf makes me think we stopped evolving too soon

#130

Post by chubot »

justahuman74 wrote: post #59
chubot wrote: post #55

Earlier quoted context omitted.

Shells have some of the absolutely bonkersly dumb parsers and interpreters Oils fixes all this, it's a from-scratch reimplementation of shell with an upgrade path from OSH (compatible) to YSH (new). --- aliases that can override keywords and change the parsing of shell script You can control this with shopt -u expand_aliases in bash, and YSH turns it off by default. Oils is statically parsed: How To Parse Shell Like…

I'm pretty excited about YSH As a heavy user of bash I'm fully aware of how unaware I am of everything that can go wrong.

Cool! If you use and know bash, right now OSH with

    shopt -s strict:all
will prevent almost all of the "silent footguns".

On the other hand, YSH has strict:all and then a bunch of NEW features.

There's been good feedback recently, which has led to many concrete changes. So your experience can definitely influence the language! https://github.com/oilshell/oil/wiki/Where-To-Send-Feedback

Post reply on HN