I agree about Forth being a fatally flawed language with superpowers, although I think we could easily have ended up in a world where Forth played the role of C, which has its own fatal flaws.
Yes, compile-time metaprogramming is very much "how things are done". This is simplified by not having syntax, but I don't think they're inseparable; you could imagine building up a compiler in the same way from an almost-as-minimal base using something like https://jevko.org/, S-expressions, a Prolog-like extensible infix parser, or a Smalltalk-like non-extensible infix parser with an open set of operators. I think most of these would be improvements. PostScript has an only slightly more elaborate syntax than Forth, but uses Smalltalk-style lightweight lambdas (called "quotations" in several other stack languages) to provide control-flow operators through runtime metaprogramming instead of compile-time metaprogramming.
As for "mechanisms used for disparate purposes", for example, the outer (text) interpreter in typical Forths plays the role of the Unix shell, the C-level systems programming language, the assembler syntax, and the user interface to applications such as, traditionally, the interactive text editor. And in https://news.ycombinator.com/item?id=45340399 drivers99 reports using it to parse an input file. The Forth language is not a very good shell command language, not a very good high-level programming language, and not a very good text editor user interface language, but it's adequate for all of these purposes.
The dictionary, similarly, serves to hold definitions for all those purposes. But it also allocates memory in a region-allocator-like way—a byte at a time, if need be. You can use the same words like , to store data into the dictionary directly, in interpretation state:
create myarray 3 , 4 , x ,
Or in a constructor:
: throuple create , , , ; 3 4 x throuple myarray
In traditional Forths like F83,
, is also the mechanism for adding an xt to a colon definition, but in ANS Forth
compile, was added as a possible synonym which would also permit writing Forth code that was portable to non-threaded-code implementations.
https://forth-standard.org/standard/core/COMPILECommaThe operand stack serves to pass arguments and return return values, as well as to hold temporaries, but you can often use it to store a local variable as well, and space on it is dynamically allocated, so it's possible to use it to pass or return variable-sized arrays by value. At compile time, it's used to keep track of the nesting of control-flow structures.
The return stack serves to store return addresses, but also to store loop counters or maybe another local variable. And return-stack manipulation provides you with a relatively flexible form of runtime metaprogramming for things like stackless coroutines, shallow-bound dynamic scoping, and exception handling. Here's an implementation of dynamic scoping (which cannot be used inside a do loop or when you have other stuff on the return stack):
0 value old 0 value where : co 2r> >r >r ;
: let! dup to where where @ to old ! co old where ! ;
Example usage:
decimal : dec. 10 base let! . ;
This temporarily sets
base to 10 before calling
., but then restores
base to whatever value it had before upon return. A better implementation that uses the return stack instead of
old and
where to save and restore the values is
: (let!) dup @ over swap 2r> rot >r rot >r >r >r ! ; : let! (let!) 2r> ! ;
(This is probably not very understandable, but I've written an 1800-word explanation of it elsewhere which you can read if you like.)
Pointer arithmetic and integer arithmetic are the same operation, as they are in most untyped languages. This is different from C, where they are done with the same operators which are implemented differently for integers and for different types of pointers.
The "filesystem" in traditional Forths simply exposes the disk as an array of 1024-byte blocks which could be mapped into memory on demand. Conventionally you would divide your code into 1024-byte screenfuls, each space-padded out to 64-character lines, 16 of them. In effect, each screen was a different "file", identified by number rather than name. It's reasonable to argue that this is not a very good filesystem, and not a very good format for text files, but to implement any filesystem on top of a disk or SSD, you need a layer that more or less provides that functionality; all that's required to make it usable for code blocks is to use 1024-byte blocks instead of 128-byte or 512-byte or whatever.
Multitasking in traditional Forths is cooperative. In some sense this eliminates the need for locking; for example, to ensure that the block buffer you've mapped your desired block into doesn't get remapped by a different task before you're done using it, you simply avoid calling anything that could yield. Unfortunately, Forth doesn't have colored functions, so there's no static verification that you didn't call anything that calls something that yields. Cooperative multitasking is sort of not very good multitasking (since an infinite loop in any task hangs the system) and not very good locking, but it does serve both purposes well enough to be usable.
Scheme is sort of like this too; famously, Scheme's lambda (roughly Forth's create does>) is semantically an OO object, a statement sequencing primitive, a lazy-evaluation primitive, etc., while S-expressions are a similar syntactic cure-all, and call/cc gives you multithreading, exception handling, backtracking, etc. See https://research.scheme.org/lambda-papers/. In practice a small Lisp is about the same amount of code as a small Forth.
BTW, I still have a paper of yours in my queue to read!