Live data from Hacker News

Pnut: A C to POSIX shell compiler you can trust

pnut.sh

111–120 of 124 posts

Re: Pnut: A C to POSIX shell compiler you can trust

#111

Earlier quoted context omitted.

It's not about performance, it's about viability. If the result is so slow that it's unusable, it doesn't matter how portable it ends up being.

We haven't found this to be an issue for Pnut. One of the metric we use for performance is how much time it takes to bootstrap Pnut, and dash takes around a minute which is about the time taken by bash. This is with Pnut allocating around 150KB of memory when compiling itself, showing that Dash can still be useful even when hundreds of KBs are allocated. One thing we did notice is that subshells can be a bottleneck w…

> We haven't found this to be an issue for Pnut. One of the metric we use for performance is how much time it takes to bootstrap Pnut, and dash takes around a minute which is about the time taken by bash. This is with Pnut allocating around 150KB of memory when compiling itself, showing that Dash can still be useful even when hundreds of KBs are allocated.

Interesting. When you say "even when hundreds of KBs are allocated", do you mean this is allocating variables with large values, or tons of small variables? My case was the latter, and with that I saw a noticeable slowdown on Dash.

Simplest repro case:

  $ cat many_vars_bench.sh
  #!/bin/sh
  
  _side=500
  
  i=0
  while [ "${i}" -lt "${_side}" ]; do
    j=0
    while [ "${j}" -lt "${_side}" ]; do
      eval "matrix_${i}_${j}=$((i+j))" || exit 1
      : $(( j+=1 ))
    done
    i=$((i+1))
  done
  
  $ time bash many_vars_bench.sh
  5.60user 0.12system 0:05.78elapsed 99%CPU (0avgtext+0avgdata 57636maxresident)k
  0inputs+0outputs (0major+13020minor)pagefaults 0swaps
  
  $ time dash many_vars_bench.sh
  40.75user 0.14system 0:41.22elapsed 99%CPU (0avgtext+0avgdata 19972maxresident)k
  0inputs+0outputs (0major+4951minor)pagefaults 0swaps
Dash was ~8 times slower. Increase the side of the square "matrix" for a proportionally bigger slowdown (this one uses 250003 variables).

> One thing we did notice is that subshells can be a bottleneck when the environment is large, and so we avoided subshells as much as possible in the runtime library. Did you observe the same in your testing?

Yes, launching a new process is generally expensive and so is spawning a subshell. If the shell is something like Bash (with a lot of startup/environment setup cost) then you'll feel this more than something like Dash, where the whole point was to make the shell small and snappy for init scripts: https://wiki.ubuntu.com/DashAsBinSh#Why_was_this_change_made...

In my limited testing, Bash generally came out on top for single-process performance, while Dash came out on top for scripts with more use of subshells.

Re: Pnut: A C to POSIX shell compiler you can trust

#113
This is not useful if it doesn't call external libraries.

Even POSIX standard ones. Chokes on:

  #include 

  int main()  // must be (); (void) results in syntax error.
  {
    glob_t gb; // syntax error here
    glob("abc", 0, NULL, &gb);
    return 0;
  }
Nobody needs entirely self-contained C programs with no libraries to be turned into shell scripts; Unix people switch to C when there is a library function they need to call for which there no command in /bin or /usr/bin.

If I reduce it to:

  #include 

  int main()
  {
    glob("abc", 0, NULL, 0);
    return 0;
  }
it "compiles" into something with a main function like:

  _main() {
    defstr __str_0 "abc"
    _glob __ $__str_0 0 $_NULL 0
    : $(($1 = 0))
  }
but what good is that without a definition of _glob.

Re: Pnut: A C to POSIX shell compiler you can trust

#114

"Because Pnut can be distributed as a human-readable shell script (`pnut.sh`), it can serve as the basis for a reproducible build system. With a POSIX compliant shell, `pnut.sh` is sufficiently powerful to compile itself and, with some effort, [TCC]( https://bellard.org/tcc/ ). Because TCC can be used to bootstrap GCC, this makes it possible to bootstrap a fully featured build toolchain from only human-readable sourc…

Problem is:

- a shell is required, which has to be built from sources, using a compiler which was also built from sources using a compile binary. That's the real boostrap.

- even if you could pick some shell, and compiled it with pnut.exe, the compiled code requires interpretation by an executable shell.

- there is no such thing as a "POSIX compliant shell"; that's an abstract category. All this amounts to is a promise that pnut.sh will not generate code that uses non-POSIX features.

Re: Pnut: A C to POSIX shell compiler you can trust

#116
post #74

Earlier quoted context omitted.

head , read , and sed can be used for seeking forward according to POSIX (see the INPUT FILES section here https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V... >). I doubt non-GNU implementations support it though.

I think dd might be more reliable. (Is dd POSIX?)

Yep: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/d...

Re: Pnut: A C to POSIX shell compiler you can trust

#117
post #110

Earlier quoted context omitted.

I think you're confusing the language and the perception of language, the "rules of C" vs. the "brand of C". What Pnut shows us is that the language itself is a very thin construct. C could be as low-level as you want, but it can also... compile to shell. Pnut shows that C is only a set of grammatical rules, and the source code in C doesn't necessary reflect the binary program, it's only a script for the C compiler.…

> The difference between: > set(SOME_VARIABLE "SOME VALUE") > and > set(SOME_VARIABLE, "SOME VALUE"); > is purely grammatical. The underlying functionality is the same. But in a build script you don't want to be doing either. You want SOME_VARIABLE = SOME VALUE, or at most "SOME VALUE". Grammar and syntax matter. > Pnut shows that C is only a set of grammatical rules, and the source code in C doesn't necessary reflec…

If syntax mattered that much, CMake would have opted for SOME VARIABLE = SOME VALUE. But... they went for set(SOME_VARIABLE "SOME VALUE") instead. I don't know why.

Syntax-wise C is fine. I personally have a soft spot for Rebol's "syntax free" approach, but the world prefers C. Five out of ten TIOBE's most popular languages have C-like syntax.

And you're right that the perception of C comes from the usage of C. Of course it does. But this creates the vicious cycle, the cycle things like Pnut are trying to break.

Re: Pnut: A C to POSIX shell compiler you can trust

#118
post #49
post #47

Earlier quoted context omitted.

> I cant run cosmolibc on Android, for example. You can: https://justine.lol/cosmo3/ > After nearly one year of development, I'm pleased to announce our version 3.0 release of the Cosmopolitan library. [...] we invented a new linker that lets you build fat binaries which can run on these platforms: AMD ... ARM64 https://github.com/jart/cosmopolitan/releases/tag/3.5.3 > This release fixes Android support. You can now…

Thanks for the link! My comment was based on cloning master yesterday and trying to build redbean but hitting what looks like https://github.com/jart/cosmopolitan/issues/940 Indeed it lioks like the commit you mentioned should have fixed the issue with the pointer having too many bits for the weird kernel used on android and some raspis. Fingers crossed that release works. edit: Testing that release on Termux 118, st…

That issue was fixed last month. I've freshened up the cocmd binary for you! https://github.com/jart/cosmopolitan/commit/e18fe1e1127f30db...

Re: Pnut: A C to POSIX shell compiler you can trust

#119
post #118
post #49

Earlier quoted context omitted.

Thanks for the link! My comment was based on cloning master yesterday and trying to build redbean but hitting what looks like https://github.com/jart/cosmopolitan/issues/940 Indeed it lioks like the commit you mentioned should have fixed the issue with the pointer having too many bits for the weird kernel used on android and some raspis. Fingers crossed that release works. edit: Testing that release on Termux 118, st…

That issue was fixed last month. I've freshened up the cocmd binary for you! https://github.com/jart/cosmopolitan/commit/e18fe1e1127f30db...

Awesome, thanks!

Re: Pnut: A C to POSIX shell compiler you can trust

#120
post #110

Earlier quoted context omitted.

> The difference between: > set(SOME_VARIABLE "SOME VALUE") > and > set(SOME_VARIABLE, "SOME VALUE"); > is purely grammatical. The underlying functionality is the same. But in a build script you don't want to be doing either. You want SOME_VARIABLE = SOME VALUE, or at most "SOME VALUE". Grammar and syntax matter. > Pnut shows that C is only a set of grammatical rules, and the source code in C doesn't necessary reflec…

If syntax mattered that much, CMake would have opted for SOME VARIABLE = SOME VALUE. But... they went for set(SOME_VARIABLE "SOME VALUE") instead. I don't know why. Syntax-wise C is fine. I personally have a soft spot for Rebol's "syntax free" approach, but the world prefers C. Five out of ten TIOBE's most popular languages have C-like syntax. And you're right that the perception of C comes from the usage of C. Of co…

> the world prefers C. Five out of ten TIOBE's most popular languages have C-like syntax.

I don't know which five you're classifying that way, but even for languages that started off C-like the trend is in the direction of less C-like. Even for C++ the big popular changes recently have been things like auto; similarly for Java, and C# always had a more lightweight syntax for expressing values. And certainly JavaScript has an object literal syntax good enough that people use it separately. Python is admittedly weirdly bad for writing values in; I wonder if that's why Scons has more or less failed.

Post reply on HN