Easy Forth (2015)

Reading lines from a file and handling the strings in memory is what made me stop using it after a 3rd day of advent of code one year. I simply couldn't find a good solution, without a massive excursion into how to use the pad. Such a supposedly simple thing like reading a complete line from a file, yet it stopped me completely. Of course I could have "cheated" and put the input right into the program, but I wanted to learn Forth, so I thought I should be able to do this ...

Later I read, that GForth 1.0 should have more string handling words, but then I already had lost hope to find an easy solution. Don't get me wrong, learning the little bit of Forth that I did learn, it was quite interesting, and I would have liked to progress more. I think I also lost hope, because I couldn't see how this stack system would ever be able to handle multi-core and persistent data structures. Things that I have come to use in other niche languages. Also that some projects/libraries are one-man shows/bus factor 1, and the maintainers have stopped developing them. They are basically stale and made by people, which significantly more understanding than any beginner will have for a long time.

I guess to really learn it, one has to read one of the often recommended books and have a lot of patience, until one gets to any parts, where one learns simple things like reading a file line by line.

Most languages don't have an explicit stack, and even their implicit stack is only for subroutine calls. If you're not making subroutine calls, your compiled code might not access the stack at all. So, for example, here's the strlcpy function from OpenBSD, lightly edited:

    size_t strlcpy (char *dst, const char *src, size_t siz) {
            register char *d = dst;
            register const char *s = src;
            register size_t n = siz;

            if (n != 0 && --n != 0) {
                    do { if ((*d++ = *s++) == 0) break; } while (--n != 0);
            }

            if (n == 0) {
                    if (siz != 0) *d = '\0';
                    while (*s++)
                            ;
            }

            return(s - src - 1);
    }

GCC 12.2.0 compiles this to the following 18 ARM instructions, with -mcpu=cortex-a53 -Os -S:

            .text
            .align 2
            .global strlcpy
            .syntax unified
            .arm
            .type strlcpy, %function
    strlcpy:
            @ args = 0, pretend = 0, frame = 0
            @ frame_needed = 0, uses_anonymous_args = 0
            @ link register save eliminated.
            mov r3, r1
            cmp r2, #0
            beq .L6
    .L14:
            subs r2, r2, #1
            beq .L3
            ldrb ip, [r3], #1 @ zero_extendqisi2
            strb ip, [r0], #1
            cmp ip, #0
            bne .L14
    .L4:
            sub r0, r3, r1
            sub r0, r0, #1
            bx lr
    .L3:
            mov r2, #0
            strb r2, [r0]
    .L6:
            ldrb r2, [r3], #1 @ zero_extendqisi2
            cmp r2, #0
            bne .L6
            b .L4
            .size strlcpy, .-strlcpy

If you're not familiar with ARM assembly, I'll tell you that nothing in this entire function uses the stack at all, which is possible because strlcpy doesn't call any other functions (it's a so-called "leaf subroutine", also known as a "leaf function") and because ARM, like most RISCs, puts the subroutine return address in a register (lr) instead of on the stack like amd64, or in the called subroutine like the PDP-8, which doesn't have a stack at all. And the calling convention puts arguments and return values in registers as well. So the function can just move data around between memory and registers and decrement its loop counter and increment its pointers without ever touching the stack.

FORTRAN up to FORTRAN 77 didn't support recursion, including indirect recursion, so that you could implement it without a stack.

By contrast, in Forth, instead of registers you use the operand stack. For loop counters you use the return stack. Sometimes you can use the operand stack instead of variables as well, although I think it's usually a better idea to use variables, especially when you're starting to learn Forth—it's much easier for beginners to get into trouble by trying too hard to use the stack instead of variables than to get into trouble by trying too hard to use variables instead of the stack.

Well said. I love Forth an I think it’s worth learning, but almost nobody programs workstation-level applications with it, and as you say, even in embedded environments the level of resources have grown such that there’s very little reason to choose Forth anymore. Which makes me a bit sad because Forth is brilliant.

I think mostly learning Forth is like learning any other programming language (or, better said, programming environment): you learn by doing it. Books can be a useful complement to practice, but practice is how you learn to do things. You can't learn to do things by reading.

As for string handling, in my limited experience, string handling in Forth is a lot like string handling in C; you have to allocate buffers and copy characters between them. memcpy is called move, and memset is called fill. You can use the pad if you want, but you can just as well create inbuf 128 allot and use inbuf. There are two big differences:

1. Forth doesn't have NUL-terminated strings like C does, because it's just as easy to return a pointer and a length from a subroutine as it would be to return just a pointer. This is generally a big win, preventing a lot of subtle and dangerous bugs. (Forth is generally more error-prone than C, but this is an exception.)

2. Forth unfortunately does have something called a "counted string", where the string length is stored in the byte before the string data. You can create them with C" (https://forth-standard.org/standard/core/Cq), and Forth beginners often wonder whether to use counted strings. The answer is no: you should never use counted strings, and they should not have been included in the standard. Use normal strings, created with S" (https://forth-standard.org/standard/core/Sq), unless you are calling word or find. https://forth-standard.org/standard/rationale#rat:cstring goes into some of the history of this.

If you want to allocate strings on the heap, which is often the simplest way to handle strings, malloc is called allocate, realloc is called resize, and free is called free: https://forth-standard.org/standard/memory

With respect to multicore and persistent data structures (I assume you mean FP-persistent, as in, an old pointer to a data structure is a pointer to the old version of the data structure), stacks aren't really related to them. Each Forth thread has its own operand stack and its own return stack (and sometimes its own dictionary), so they don't really create interactions between different cores.

If I can find a Forth for a new chip I'm learning, I use it as a way to explore the chip with a REPL.

Micropython can be similar this way, but it's more constrictive.