The naked truth about writing a programming language (2014)

(Not Walter but) Self-hosting in a compiler should be done wherever possible - if we ignore that if the compiler self-hosts it becomes it's own test suite (writing tests is good but it's quite difficult to find weird behavioural bugs using unittests if the tests only test one thing at a time) - it makes it so much easier to work on the compiler (For example, the main D compiler can build itself in a second or two on my machine - if I had to cart around some other huge toolchain it would be much slower). Another boon is that, if people who are really good (let's say) Go programmers want to make the Go compiler better they then don't have to try and write go in C++ if the main compiler is written in C++.

Also, it can also be a plan to rush to a MVP, rewrite the MVP in your language and go from there. This has the added benefit of giving you a decent sized program in your language at no extra cost - aside from testing this should make you design a better language as you learn what works at what doesn't in the "real world".

It's not always possible (Please don't write a compiler in Javascript, or even C if possible - pain and lack of abstraction respectively).

That's a good set of questions for 2014. Questions that have become important more recently include:

- Imperative? Functional? Some mixture of both? Mixtures of the two tend to have syntax problems.

- Concurrency primitives. The cool kids want "async" now. Mostly to handle a huge number of slow web clients from one server process. Alternatively, there are "green threads", which Go calls "goroutines". All this stuff implies some level of CPU dispatching in compiled code.

- Concurrency locking. Really hard problem. There's the Javascript run to completion approach, which reappears in other languages as "async". Ownership based locking, as in Rust, is promising, but may lock too much. And what about atomic operations and lockless operations? Source of hard to find bugs. Get this right.

- Ownership. Rust got serious about this, with the borrow checker. C++ now tries to do "move semantics", with modest success, but has trouble checking at compile time for ownership errors. Any new language that isn't garbage collected has to address this. Historically, language design ignored this problem, but that's in the past.

- Metaprogramming. Templates. Generics. Capable of creating an awful mess of unreadable code and confusing error messages. From LISP macros forward, a source of problems. Needs really good design, and there aren't many good examples to follow.

- Integration with other languages. Protocol buffers? SQL? HTML? Should the language know about those?

- GPUs. Biggest compute engines we have. How do we program them?

In some programming languages, e.g. TeX, Forth, PostScript, etc you cannot really syntax highlight the program without executing it. However, syntax highlighting is a feature that I can do without, and I don't think those programming languages are bad. In some programming languages, such as C and Free Hero Mesh, you can read the sequence of tokens despite there are macros, which you need not parse to find where the tokens are (although I think older versions of the C preprocessor did not work like this).

They mention tools. I do think valgrind is sometimes helpful; I do use that. However, Git is not the only version control system; some people prefer others, such as Mercurial, Fossil, etc. I use Fossil.

They also mention error messages. I like the first one; print one error message and quit. However, in some cases, it might be possible to easily just ignore that error and continue after displaying the error message (and then finally fail at the end). It might also be possible to skip a lot of stuff, and then continue, e.g. if there is an unknown variable or function then you might display an error message and then skip the entire statement that mentions it.

They mention a runtime library. This should be reduced as much as possible, I think, and if you can make it so that it has functions which will only be included in the compiled program when they are used, that can be a good idea, since you can include some functions that many people don't need and don't waste space and time with them.

Something they did not mention but I think it is very good to have and useful is metaprogramming and 'pataprogramming.

But I also think that different programming languages can be good for different purposes, and that some are domain specific, and some are easier to write than others. This can be done using existing syntax or new syntax, and I have done both, and so have some others.

Another thing that I can recommend to have is a discussion system with NNTP; even Digital Mars themself have a NNTP-based discussion system. Perhaps better will be to have the same messages with NNTP, web forum, and mailing lists; again, this is what Digital Mars does.

However, I do think that minimizing keystrokes is helpful. Minimizing runtime dependencies and runtime memory usage is also helpful.

I would say possibly to use a different programming language for GPU than CPU. I like the idea of Checkout (see [0]) for GPU programming, although unfortunately it is not implemented and the preprocessor is not yet invented. (The trigonometric functions are also missing. They should probably have at least sine, cosine, and arctangent, since these functions seem like useful to me when doing graphics.)

[0] http://esolangs.org/wiki/Checkout

Thank you very much for the references. They are very good reads!

In the article about Python, the author said:

> Most languages of interest are context sensitive, and yet we are trying to use inappropriate tools (context-free grammars and context-free parser generators) to specify and parse them. This leads to bad specifications and bad implementations.

Then what do you think is a better tool to specify and parse languages?