Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autoconf makes me think we stopped evolving too soon

rachelbythebay.com

Post reply on HN
Print view View original

51–60 of 169 posts

Re: Autoconf makes me think we stopped evolving too soon

#51

Post by TillE »

opello wrote: post #34
elicksaur wrote: post #25

Is anyone doing and publicizing the git clone vs. tarball work in other projects? I suppose if something has been found, it might be too soon to disclose publicly.

The major downside of using git clone build workflows from an autotools project is that your build environment and the developer's may have different enough versions of autotools that macro incompatibility introduces more problems for you to solve. I've noticed this less recently (say 5 years) but still has come up occasionally. While a distributed tarball will have the "developer blessed" autoreconf run and declared…

Ultimately autotools isn't doing anything magical, so it's not unusual for cross-platform package managers like vcpkg to simply ignore all that and write their own clean CMakeLists.txt.

In this particular case xz-utils actually did support CMake for building liblzma, so that's what you use if you don't care about the command line tools.

Re: Autoconf makes me think we stopped evolving too soon

#52

Post by chubot »

nabla9 wrote: post #36
titzer wrote: post #32

Autoconf is one of the absolutely hilarious things about UNIX. On the one hand, we've got people optimizing kernels down to the individual instructions (often doing very unsafe dirty C tricks underneath), sometimes with super-clunky and overly complex APIs as a result...and on the other hand you have all the shell script absolute nuttery like the behemoth heap of kludges that is autoconf. It's crazy to me the disconn…

Alan Perlis Epigram #9: "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures." Unix settled for string as a data structure and it's little too low level. All Lisp programmers know that s-expressions would have been the right choice. JSON works also.

Yup, I call that the Perlis-Thompson Principle -- because Ken Thompson made a similar combinatorial argument about software composition: you should design it around "one thing".

https://www.oilshell.org/blog/2021/07/blog-backlog-1.html#co...

Files had structure on pre-Unix OSes, but they don't on Unix, because it doesn't compose.

The "Uniform Interface Constraint" of REST is the same thing, and resembles a file system -- everything is GET / POST on URLs.

---

So the reason that shell/Unix and HTTP are so common is a MATHEMATICAL property of software growth.

How do you make a Zig program talk to a Mojo program? Probably with a byte stream.

What about a Clojure program and a Common Lisp program? Probably a byte stream. (Ironically, S-expressions have no commonly used "exterior" interchange format)

Every time a new language is introduced, I think "well there's another reason you're going to need a shell script".

---

The larger the system, the more heterogeneous it is. And software is larger now, which is why MORE GLUE is needed.

This is why shell was the #6 fastest growing language on Github in 2022: https://octoverse.github.com/2022/top-programming-languages

And the #1 fastest growing language is HCL, which is a very closely related form of glue. It's basically "shell for distributed systems"

---

https://www.oilshell.org/ has JSON as of a few months ago, and is now pure native code (no more Python)

All the normal shell stuff works:

    osh$ ls */*.py | wc -l; whoami
    327
    andy
But you can seamlessly create dicts and lists like JavaScript. They are fully GC'd data structures:

    osh$ var d = {name: 42}
And then you can write them as JSON:

    osh$ json write (d)
    {
      "name": 42
    }
And you can also upgrade to ysh with one line:

    shopt --set ysh:upgrade
(YSH isn't stable yet, but OSH is very stable)

I think of the data languages as layered -- i.e. an entire json message literally is a string, and it can also contain strings:

https://www.oilshell.org/blog/2023/06/ysh-design.html#exteri...

Re: Autoconf makes me think we stopped evolving too soon

#53

Post by cpeterso »

db48x wrote: post #28
thechao wrote: post #13

Earlier quoted context omitted.

I'm be written lots of C code that runs on lots of architectures & platforms: PDPs, x86, ARM; Windows, BSD, Linux, VMS. My projects have spanned them gamut from tiny to gargantuan (millions of lines of code). I've never needed autoconf... what's it even for !?

In decades past, if you were developing software for Unix you would inevitably run across compatibility problems. Even with the POSIX standard, every Unix was trying new things, inventing new things, or just implementing the same thing in slightly different ways. So your programs had to have a bunch of macros to switch between different variations. Some API would have two arguments on Unix A but three on Unix B, so y…

It always seemed to me that most of these differences and macros could be defined in a static header file that has a million #ifdefs for different OS and compiler versions. That seems simpler and more reliable, and usable for cross compiling, than running dynamic checks with autoconf.

Re: Autoconf makes me think we stopped evolving too soon

#54

Post by theamk »

forrestthewoods wrote: post #14

A huge part of the problem is relying on the system. Project repos should include their dependencies. The Linux model of global shared libraries is, imho, bad and wrong. Building a project shouldn’t be complicated. There should be extremely minimal branching and if checks. My blog post where I argue this in more detail: https://www.forrestthewoods.com/blog/dependencies-belong-in-...

Something does not compute... problem:

> Have you ever had a build fail because of a network error on some third-party server? Commit your dependencies and that will never happen.

author's "ideal" solution:

> First, the user clones a random GenAI repo. This is near instantaneous as files are not prefetched. The user then invokes the build script. As files are accessed they're downloaded. The very first build may download a few hundred megabytes of data.

Seems like contradiction to me? Do you want your builds to be offline or online?

I have a feeling that the real problem is that (1) author never worked with build systems which supply their own compiler, like buildroot or hermetic bazel and (2) author only downloaded dependencies from third-party servers.

You can solve all the problems in the blog post by (1) changing build system to download compilers and libraries and (2) adopting some sort of "binary storage" (something as simple as private ftp server + small text file checked into repo with path and checksum + few scripts to upload and download). And it would not require any new major code investment, nor new tooling, nor new VCS.

The key idea is that all dependencies should always come from your local binary server, never from the third-party server. This is a bit of PITA as many modern tools default to downloading stuff from third-party servers, and it could be hard to convince them to look at your local server instead. But once you do it, you get reliability + reproducibility + sustainability, without having to reinvent the world.

Re: Autoconf makes me think we stopped evolving too soon

#55

Post by chubot »

titzer wrote: post #32

Autoconf is one of the absolutely hilarious things about UNIX. On the one hand, we've got people optimizing kernels down to the individual instructions (often doing very unsafe dirty C tricks underneath), sometimes with super-clunky and overly complex APIs as a result...and on the other hand you have all the shell script absolute nuttery like the behemoth heap of kludges that is autoconf. It's crazy to me the disconn…

Shells have some of the absolutely bonkersly dumb parsers and interpreters

Oils fixes all this, it's a from-scratch reimplementation of shell with an upgrade path from OSH (compatible) to YSH (new).

---

aliases that can override keywords and change the parsing of shell script

You can control this with shopt -u expand_aliases in bash, and YSH turns it off by default.

Oils is statically parsed: How To Parse Shell Like a Programming Language - https://www.oilshell.org/blog/2019/02/07.html

---

look up how "[" and "]" is handled in some shells--it might not be!

OSH runs POSIX [ as-is, for compatibility:

    osh$ [ x = x ] && echo yes
    yes
But YSH nudges you toward a "saner" style:

    ysh$ [ x = x ] && echo yes
      [ x = x ] && echo yes
      ^
    [ interactive ]:2: '[' should be invoked as 'test' (simple_test_builtin)
It also gives you a more familiar syntax. It's for Python and JavaScript users who avoid shell:

    ysh$ if ('x' === 'x') { echo yes }
    yes
---

A Tour of YSH - https://www.oilshell.org/release/latest/doc/ysh-tour.html

Why Create a New Shell? - https://www.oilshell.org/blog/2021/01/why-a-new-shell.html

(OSH is very stable and compatible, but YSH isn't stable yet. Getting there)

Re: Autoconf makes me think we stopped evolving too soon

#56

Post by drewg123 »

jonstewart wrote: post #35
amluto wrote: post #23

Earlier quoted context omitted.

libtool does serve a purpose: it causes plain 'make' to produce shell script wrappers as outputs, which is maybe kinda sorta vaguely useful when developing and utterly and completely obnoxious if you want an actual ELF file. So you end up running 'make install' when you didn’t actually want to install or trying to remember how to find the actual ELF files. And you then wonder why 'make install' appears to be building…

Oh right, how could I forget that awful aspect of using libtool? I hate my life; maybe 2024 will be the year I find a decent build tool and the motivation to move projects to it. Contra your slight defense, is it really so hard to produce a good DSO these days? -fPIC is half the battle and a couple other weird flags for macOS/darwin will do you. Cross-compiling for Windows, sure, a few more flags — but you look them…

100% this. Libtool is awful.

In the early 2000s, I ripped libtool out of a product that built on Linux, Solaris, FreeBSD, Aix, Tru64, Mac OSX and Windows.

Re: Autoconf makes me think we stopped evolving too soon

#57

Post by bhawks »

forrestthewoods wrote: post #14

A huge part of the problem is relying on the system. Project repos should include their dependencies. The Linux model of global shared libraries is, imho, bad and wrong. Building a project shouldn’t be complicated. There should be extremely minimal branching and if checks. My blog post where I argue this in more detail: https://www.forrestthewoods.com/blog/dependencies-belong-in-...

As a Xoogler one of the things I miss the most was srcfs/CitC(client in the cloud). It worked exactly as you were describing and also brought some very useful features including transparent read access to other users clients and time machine like capabilities. After having every single edit saved for you doing period git commits feels like such a chore.

Re: Autoconf makes me think we stopped evolving too soon

#58

Post by amluto »

jonstewart wrote: post #35
amluto wrote: post #23

Earlier quoted context omitted.

libtool does serve a purpose: it causes plain 'make' to produce shell script wrappers as outputs, which is maybe kinda sorta vaguely useful when developing and utterly and completely obnoxious if you want an actual ELF file. So you end up running 'make install' when you didn’t actually want to install or trying to remember how to find the actual ELF files. And you then wonder why 'make install' appears to be building…

Oh right, how could I forget that awful aspect of using libtool? I hate my life; maybe 2024 will be the year I find a decent build tool and the motivation to move projects to it. Contra your slight defense, is it really so hard to produce a good DSO these days? -fPIC is half the battle and a couple other weird flags for macOS/darwin will do you. Cross-compiling for Windows, sure, a few more flags — but you look them…

> Contra your slight defense, is it really so hard to produce a good DSO these days? -fPIC is half the battle

I did say “slight” for a reason :)

Re: Autoconf makes me think we stopped evolving too soon

#59

Post by justahuman74 »

chubot wrote: post #55
titzer wrote: post #32

Autoconf is one of the absolutely hilarious things about UNIX. On the one hand, we've got people optimizing kernels down to the individual instructions (often doing very unsafe dirty C tricks underneath), sometimes with super-clunky and overly complex APIs as a result...and on the other hand you have all the shell script absolute nuttery like the behemoth heap of kludges that is autoconf. It's crazy to me the disconn…

Shells have some of the absolutely bonkersly dumb parsers and interpreters Oils fixes all this, it's a from-scratch reimplementation of shell with an upgrade path from OSH (compatible) to YSH (new). --- aliases that can override keywords and change the parsing of shell script You can control this with shopt -u expand_aliases in bash, and YSH turns it off by default. Oils is statically parsed: How To Parse Shell Like…

I'm pretty excited about YSH

As a heavy user of bash I'm fully aware of how unaware I am of everything that can go wrong.

Re: Autoconf makes me think we stopped evolving too soon

#60

Post by wyldfire »

I'm finding myself in the unenviable position of defending autotools.

> They were a reflection of the way the kernel, C libraries, APIs and userspace happened to work. Short of that changing, the results wouldn't change either.

These things do change, especially as toolchain and OS vendors realize their quirks that cause portability friction.

Post reply on HN