Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pnut: A C to POSIX shell compiler you can trust

pnut.sh

Post reply on HN
Print view View original

101–110 of 124 posts

Re: Pnut: A C to POSIX shell compiler you can trust

#101

Post by laurenth »

osmsucks wrote: post #52
n_plus_1_acc wrote: post #48

Earlier quoted context omitted.

I hope you're not compiling C to sh for performance reasons.

It's not about performance, it's about viability. If the result is so slow that it's unusable, it doesn't matter how portable it ends up being.

We haven't found this to be an issue for Pnut. One of the metric we use for performance is how much time it takes to bootstrap Pnut, and dash takes around a minute which is about the time taken by bash. This is with Pnut allocating around 150KB of memory when compiling itself, showing that Dash can still be useful even when hundreds of KBs are allocated.

One thing we did notice is that subshells can be a bottleneck when the environment is large, and so we avoided subshells as much as possible in the runtime library. Did you observe the same in your testing?

Re: Pnut: A C to POSIX shell compiler you can trust

#102

Post by westurner »

Cloudef wrote: post #83
theamk wrote: post #19

If you are wondering how it handles C-only functions.. it does not. open(..., O_RDWR | O_EXCL) -> runtime error, "echo "Unknow file mode" ; exit 1" lseek(fd, 1, SEEK_HOLE); -> invalid code (uses undefined _lseek) socket(AF_UNIX, SOCK_STREAM, 0); -> same (uses undefined _socket) looking closer at "cp" and "cat" examples, write() call does not handle errors at all. Forget about partial writes, it does not even return -…

There seems to be libc in the repo but many functions are TODO https://github.com/udem-dlteam/pnut/tree/main/portable_libc Otherwise the builtins seems to be here https://github.com/udem-dlteam/pnut/blob/main/runtime.sh FYI all your functions are not "C functions", but rather POSIX functions. I did not expect it to be complete, but it's still impressive for what it is.

There are Linux ports of the plan9 `syscall` binary, which is presumably necessary to implement parts of libc with shell scripts: https://stackoverflow.com/questions/10196395/os-system-calls...

I don't remember there being a way to keep a server listening on a /dev/tcp/$ip/$port port, for sockets from shell scripts with shellcheck at least

Re: Pnut: A C to POSIX shell compiler you can trust

#103

Post by lmm »

okaleniuk wrote: post #51

I love things like these because they shake our perception of normal loose. And who said our perception of normal doesn't deserve a good shake? A C to shell compiler might seem impractical, but you know what is even more impractical? Having a separate language for a build system. And yet, here we are. Using Shell, Make or CMake to build a C program is only acceptable because is has always been so. It's a "perceived n…

> Using Shell, Make or CMake to build a C program is only acceptable because is has always been so.

Nah, using shell, make or cmake is acceptable because C is obviously a terrible language for doing things. (Those languages are also all terrible, but not quite as terrible as C).

> There is no good reason, however, CMake isn't a C library.

Isn't it the other way round? There's no good reason people write programs in C rather than CMake.

> With build system being a library, we could write, read, and, most importantly, debug build scripts just like any other part of the buildable.

Which is to say, with extreme difficulty?

Like, I agree with where you're coming from, it is absolutely a damning indictment of C that people don't want to express their builds in it. But writing in a build in C really would be terrible.

Re: Pnut: A C to POSIX shell compiler you can trust

#104

Post by lmm »

blueflow wrote: post #53
teo_zero wrote: post #33

Just to be clear, the input must be written in a subset of C, because many constructs are not recognized, like unsigned types, static variables, [] arrays, etc. Is there a plan to remove such limitations?

These are restrictions of the target language and there isn't much pnut can do about this.

Shell is Turing complete, you could implement anything there with enough effort.

Re: Pnut: A C to POSIX shell compiler you can trust

#105

Post by bitwize »

okaleniuk wrote: post #51

I love things like these because they shake our perception of normal loose. And who said our perception of normal doesn't deserve a good shake? A C to shell compiler might seem impractical, but you know what is even more impractical? Having a separate language for a build system. And yet, here we are. Using Shell, Make or CMake to build a C program is only acceptable because is has always been so. It's a "perceived n…

Terry Davis was right. C should be your shell, as God intended.

Re: Pnut: A C to POSIX shell compiler you can trust

#106

Post by binary132 »

eichin wrote: post #81
forrestthewoods wrote: post #24

Earlier quoted context omitted.

My rule of thumb: Shell: 500 lines Although to be honest I'd be perfectly happy if Shell was restricted to single line commands only. I've wasted a lot of time and energy deciphering undebuggable shell scripts that were written to "save programmer time". Not a fan.

My rule (and the code review policy I impose) emphasizes complexity instead - a 50 line shell script is great if it doesn't use if or case. (It's not so much of a strict rule as "once you're nesting conditionals, or using any shell construct that really needs a comment to explain the shell and not your code, you should probably already have switched to python." This is in parallel with "error handling in this case is…

Nothing is as obvious as it could be when it’s 3am and you’re debugging a production outage. :)

Re: Pnut: A C to POSIX shell compiler you can trust

#107

Post by vlovich123 »

theamk wrote: post #19

If you are wondering how it handles C-only functions.. it does not. open(..., O_RDWR | O_EXCL) -> runtime error, "echo "Unknow file mode" ; exit 1" lseek(fd, 1, SEEK_HOLE); -> invalid code (uses undefined _lseek) socket(AF_UNIX, SOCK_STREAM, 0); -> same (uses undefined _socket) looking closer at "cp" and "cat" examples, write() call does not handle errors at all. Forget about partial writes, it does not even return -…

I suspect the “trust” is a reference to Ken Thompson’s Turing Award speech “Reflections on trusting trust” where he laid out the concern of a back door in a compiler that survives updates to the compiler. In other words, the compiler injects a back door into future versions of itself in addition into your programs that source level analysis of the code will never reveal.

I think the pitch here is that it can compile TCC which can then compile GCC which makes it much more difficult for a backdoor to survive potentially, especially if the shell code is easier to read and verify than the corresponding assembly.

Within that context, an incomplete libc is irrelevant.

Re: Pnut: A C to POSIX shell compiler you can trust

#108

Post by okaleniuk »

lmm wrote: post #103
okaleniuk wrote: post #51

I love things like these because they shake our perception of normal loose. And who said our perception of normal doesn't deserve a good shake? A C to shell compiler might seem impractical, but you know what is even more impractical? Having a separate language for a build system. And yet, here we are. Using Shell, Make or CMake to build a C program is only acceptable because is has always been so. It's a "perceived n…

> Using Shell, Make or CMake to build a C program is only acceptable because is has always been so. Nah, using shell, make or cmake is acceptable because C is obviously a terrible language for doing things. (Those languages are also all terrible, but not quite as terrible as C). > There is no good reason, however, CMake isn't a C library. Isn't it the other way round? There's no good reason people write programs in C…

I think you're confusing the language and the perception of language, the "rules of C" vs. the "brand of C".

What Pnut shows us is that the language itself is a very thin construct. C could be as low-level as you want, but it can also... compile to shell. Pnut shows that C is only a set of grammatical rules, and the source code in C doesn't necessary reflect the binary program, it's only a script for the C compiler. A compiler then decides how to interpret the source and what to do with it.

Now back to builds. The difference between:

    set(SOME_VARIABLE "SOME VALUE")
and

    set(SOME_VARIABLE, "SOME VALUE");
is purely grammatical. The underlying functionality is the same. When I'm saying, CMake could be a C library, I'm not saying we should ditch CMake and everything it brings to the table and start writing build scripts in pure C. I'm saying we can use both C language and CMake functionality with very little, skin deep, adjustments.

The only thing that keeps us down is the perception of C as a low-level language for low-level applications. C is for drivers and shell is for moving files around. And that's when Pnut comes up and tells us: "hold on, are they?"

Re: Pnut: A C to POSIX shell compiler you can trust

#109

Post by okaleniuk »

thechao wrote: post #61
defrost wrote: post #56

Earlier quoted context omitted.

Both the tweezers and the bit flipping magnet .. and who would want anything more?

Yeah — C would be ok as a build system language if it was easy to: invoke & manage subprocesses; build & manage dynamic dependency graphs; and, easily work with file metadata. Or... work with me: Make does that, well enough.

And that's why I'm saying CMake should have been a library. We want the functionality of Make but not necessary the language. And Pnut shows us well that {language != functionality}.

For the sake of mental experiment, let's pretend Make is a separate executable, separate process, but with some sort of API. You can manage dynamic dependency graphs by calling its routines from C.

Now let's say Make is a dynamic library with all functionality exposed. You can invoke and manage subprocesses using its functions, but now your C program and the Make share a process together.

Now let's say Make is a C library. GNU Make is written in C so this is not impossible to imagine. Your C program shares the process, and the names on compilation+linking phase with Make, which is annoying. But you can still work with metadata using Make's facilities. Also now you can use all the tools: debuggers, profilers, static analyzers, dynamic analyzers - you use for the rest of your codebase.

We perceive C as a low-level language but, and Pnut shows it well, C is only a set of rules. We can write shell scripts with C rules. Why can't we then write build scripts?

Re: Pnut: A C to POSIX shell compiler you can trust

#110

Post by lmm »

okaleniuk wrote: post #108
lmm wrote: post #103

Earlier quoted context omitted.

> Using Shell, Make or CMake to build a C program is only acceptable because is has always been so. Nah, using shell, make or cmake is acceptable because C is obviously a terrible language for doing things. (Those languages are also all terrible, but not quite as terrible as C). > There is no good reason, however, CMake isn't a C library. Isn't it the other way round? There's no good reason people write programs in C…

I think you're confusing the language and the perception of language, the "rules of C" vs. the "brand of C". What Pnut shows us is that the language itself is a very thin construct. C could be as low-level as you want, but it can also... compile to shell. Pnut shows that C is only a set of grammatical rules, and the source code in C doesn't necessary reflect the binary program, it's only a script for the C compiler.…

> The difference between: > set(SOME_VARIABLE "SOME VALUE") > and > set(SOME_VARIABLE, "SOME VALUE"); > is purely grammatical. The underlying functionality is the same.

But in a build script you don't want to be doing either. You want SOME_VARIABLE = SOME VALUE, or at most "SOME VALUE". Grammar and syntax matter.

> Pnut shows that C is only a set of grammatical rules, and the source code in C doesn't necessary reflect the binary program, it's only a script for the C compiler.

The only thing worse than writing C is writing something that looks like C but doesn't follow the rules of C, where you have to use some other logic to understand what it actually does. Build tools that do that kind of thing have been tried and they have not turned out well.

> When I'm saying, CMake could be a C library, I'm not saying we should ditch CMake and everything it brings to the table and start writing build scripts in pure C. I'm saying we can use both C language and CMake functionality with very little, skin deep, adjustments.

"Skin deep" perhaps, but making your language uglier and weirder is still unpleasant (and CMake is unpleasant and weird enough as it is).

> The only thing that keeps us down is the perception of C as a low-level language for low-level applications.

No, the other thing is the perception of C as a crude, inexpressive language full of weird edge cases that requires dozens of lines to write even simple things, and that in turn comes from the reality of C as a crude, inexpressive language full of weird edge cases that requires dozens of lines to write even simple things.

Post reply on HN