Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pnut: A C to POSIX shell compiler you can trust

pnut.sh

Post reply on HN
Print view View original

21–30 of 124 posts

Re: Pnut: A C to POSIX shell compiler you can trust

#21

Post by laurenth »

wahern wrote: post #14

This is very cool, regardless of how serious it was intended to be taken. Before base-64 encoders/decoders became more common as preinstalled commands in the environments I found myself on, I wrote a base64 utility in mostly pure POSIX shell: https://25thandClement.com/~william/2023/base64.sh If this project had existed I might have opted to compile my C-based base-64 encoder and decoder routines, suitably tweaked fo…

One of the example we include is a base64 encoder/decoder:

  https://github.com/udem-dlteam/pnut/blob/main/examples/compiled/base64.sh
It doesn't support NULs as you pointed out, but it's interesting to see similarities between your implementation and the one generated by Pnut.

Because we use `read -r`, we haven't tested reading binary files. Fortunately, the shell's `printf` function can emit all 256 characters so Pnut can at least output binary files. This makes it possible for Pnut to have a x86 backend for the use of reproducible builds.

Regarding the use of `read`, one constraint we set ourselves when writing Pnut is to not use any external utilities, including those that are specified by the POSIX standard (other than `read` and `printf`). This maximizes portability of the code generated by Pnut and is enough for the reproducible build use case.

We're still looking for ways to integrate existing shell code with C. One way this can be done is through the use of the `#include_shell` directive which includes existing shell code in the generated shell script. This makes it possible to call the necessary utilities to read raw bytes without having Pnut itself depends on less portable utilities.

Re: Pnut: A C to POSIX shell compiler you can trust

#22

Post by binary132 »

forrestthewoods wrote: post #13
marcodiego wrote: post #11

Earlier quoted context omitted.

Master Foo once said to a visiting programmer: “There is more Unix-nature in one line of shell script than there is in ten thousand lines of C.” The programmer, who was very proud of his mastery of C, said: “How can this be? C is the language in which the very kernel of Unix is implemented!” Master Foo replied: “That is so. Nevertheless, there is more Unix-nature in one line of shell script than there is in ten thous…

And then the programmer had to debug a hundred line shell script and they realized it should have all been written in Python or Rust instead. Master Foo is shorthand for Fool.

Shell is just one way. There’s nothing that says we can’t do better than shell, but what it’s good at is saving programmer time when the need isn’t there for more, and Rust is definitely not good at that.

Re: Pnut: A C to POSIX shell compiler you can trust

#23

Post by andrewf »

jcranmer wrote: post #15
andrewf wrote: post #8

Looking forward to the point where this can build autoconf. It's great that the generated ./configure script is portable but if I want to make substantial changes to the project I need to find a binary for my machine (and version differences can be quite substantial)

> Looking forward to the point where this can build autoconf. Autoconf is a perl program that turns (heavily customized) m4 files into shell scripts. How does a C compiler help there?

> Autoconf is a perl program

Oof, did not realize.

Re: Pnut: A C to POSIX shell compiler you can trust

#24

Post by forrestthewoods »

binary132 wrote: post #22
forrestthewoods wrote: post #13

Earlier quoted context omitted.

And then the programmer had to debug a hundred line shell script and they realized it should have all been written in Python or Rust instead. Master Foo is shorthand for Fool.

Shell is just one way. There’s nothing that says we can’t do better than shell, but what it’s good at is saving programmer time when the need isn’t there for more, and Rust is definitely not good at that.

My rule of thumb:

    Shell:  500 lines
Although to be honest I'd be perfectly happy if Shell was restricted to single line commands only.

I've wasted a lot of time and energy deciphering undebuggable shell scripts that were written to "save programmer time". Not a fan.

Re: Pnut: A C to POSIX shell compiler you can trust

#25

Post by 1vuio0pswjnm7 »

"Because Pnut can be distributed as a human-readable shell script (`pnut.sh`), it can serve as the basis for a reproducible build system. With a POSIX compliant shell, `pnut.sh` is sufficiently powerful to compile itself and, with some effort, [TCC](https://bellard.org/tcc/). Because TCC can be used to bootstrap GCC, this makes it possible to bootstrap a fully featured build toolchain from only human-readable source files and a POSIX shell.

Because Pnut doesn't support certain C features used in TCC, Pnut features a native code backend that supports a larger subset of C99. We call this compiler `pnut-exe`, and it can be compiled using `pnut.sh`. This makes it possible to compile `pnut-exe.c` using `pnut.sh`, and then compile TCC, all from a POSIX shell."

Anywhere we can see a step-by-step demo of this process.

Curious if the authors tried NetBSD or OpenBSD, or using another small C compiler, e.g., pcc.

Historically, tcc was problematic for NetBSD and its forks. Not sure about today, but tcc is still in NetBSD pkgsrc WIP which suggests problems remain.

Re: Pnut: A C to POSIX shell compiler you can trust

#26

Post by shric »

forrestthewoods wrote: post #13
marcodiego wrote: post #11

Earlier quoted context omitted.

Master Foo once said to a visiting programmer: “There is more Unix-nature in one line of shell script than there is in ten thousand lines of C.” The programmer, who was very proud of his mastery of C, said: “How can this be? C is the language in which the very kernel of Unix is implemented!” Master Foo replied: “That is so. Nevertheless, there is more Unix-nature in one line of shell script than there is in ten thous…

And then the programmer had to debug a hundred line shell script and they realized it should have all been written in Python or Rust instead. Master Foo is shorthand for Fool.

Master Foo long predates Python and Rust.

Re: Pnut: A C to POSIX shell compiler you can trust

#27

Post by forrestthewoods »

shric wrote: post #26
forrestthewoods wrote: post #13

Earlier quoted context omitted.

And then the programmer had to debug a hundred line shell script and they realized it should have all been written in Python or Rust instead. Master Foo is shorthand for Fool.

Master Foo long predates Python and Rust.

Masters live to be surpassed by their students. Just because something was best in class in the 80s doesn't mean it should still be used.

Re: Pnut: A C to POSIX shell compiler you can trust

#28

Post by PhilipRoman »

theamk wrote: post #19

If you are wondering how it handles C-only functions.. it does not. open(..., O_RDWR | O_EXCL) -> runtime error, "echo "Unknow file mode" ; exit 1" lseek(fd, 1, SEEK_HOLE); -> invalid code (uses undefined _lseek) socket(AF_UNIX, SOCK_STREAM, 0); -> same (uses undefined _socket) looking closer at "cp" and "cat" examples, write() call does not handle errors at all. Forget about partial writes, it does not even return -…

Implementation issues aside, while technically it should be possible to seek a file descriptor from shell through a suitable helper program in C, I believe none of the POSIX utilities provide this facility

Re: Pnut: A C to POSIX shell compiler you can trust

#30

Post by teo_zero »

laurenth wrote: post #21
wahern wrote: post #14

This is very cool, regardless of how serious it was intended to be taken. Before base-64 encoders/decoders became more common as preinstalled commands in the environments I found myself on, I wrote a base64 utility in mostly pure POSIX shell: https://25thandClement.com/~william/2023/base64.sh If this project had existed I might have opted to compile my C-based base-64 encoder and decoder routines, suitably tweaked fo…

One of the example we include is a base64 encoder/decoder: https://github.com/udem-dlteam/pnut/blob/main/examples/compiled/base64.sh It doesn't support NULs as you pointed out, but it's interesting to see similarities between your implementation and the one generated by Pnut. Because we use `read -r`, we haven't tested reading binary files. Fortunately, the shell's `printf` function can emit all 256 characters so Pnu…

Sorry, but since the very goal of base64 is to encode "uncomfortable" bytes, saying that your example doesn't work with uncomfortable bytes is like providing a fibonacci demo that only works with arguments less than 3, or a clock that only shows correct time twice a day.

I'd choose a different example to showcase pnut.

Post reply on HN