Have you read the Copy-and-patch compilation paper? It seems broadly similar to what you're describing, and Deegen. They use GHC's calling convention for it's snippets which has only caller-saved registers and all arguments passed in registers in order to optimize stitching the snippets together (via JIT templating in it, but with tailcalls in your case would probably also be the same design space).
The copy-and-patch paper is also written by me. Deegen a follow-up work of copy-and-patch, and it will use copy-and-patch as a tool to build its JIT tiers in the future.
There's some prior art on DSLs for efficient interpreters, like Anton Ertl's vmgen. https://www.complang.tuwien.ac.at/anton/vmgen/
Nice, do you know whether this DSL was ever used successfully fir other endeavours (abstract interpretation, lowering to Why3, any kind of symbolic execution? or a language server?). I've been looking for a 'flex/bison with complete semantics' and it might be a piece of the puzzle.
I don't know -- there were at least a couple papers about it, but I haven't been following. At the time it was first announced I used some of the ideas in one of my projects (https://github.com/darius/idel/blob/master/src/opcodes.awk) but unfortunately didn't develop it further.
> More recently I have been experimenting with a new calling convention that uses no callee-saved registers to work around this, but the results so far are inconclusive. The new calling convention would use all registers for arguments, but allocate registers in the opposite order of normal functions, to reduce the chance of overlap. I have been calling this calling convention "reverse_cc". As explained in the article…
I tried exposing the "cc 10" and "cc 11" calling conventions to Clang, but was unsuccessful. With "cc 10" I got crashes at runtime I could not explain. With "cc 11" I got a compile-time error: > fatal error: error in backend: Can't generate HiPE prologue without runtime parameters If "cc 10" would work from Clang I'd be happy to use that. Though I think reverse_cc could potentially still offer benefits by ordering ar…
>More importantly, it is the world’s fastest Lua interpreter to date, outperforming LuaJIT’s interpreter by 28% and the official Lua interpreter by 171% on average on a variety of benchmarks Wait what the hell
Faster than the LuaJIT's interpreter . People who focus on JIT often focus on the JIT and not the interpreter. Which is a shame because if you make the uncommon paths cheaper then you can tune your code for the hot paths a bit more aggressively. You get fewer situations where you are sacrificing Best Case for Average Case performance.
As in, LuaJIT with the JIT disabled, ie with the `-j off` flag? That would be really good to clarify in the article. They start off talking about how they can generate an interpreter and even a JIT, LuaJIT with its JIT turned on is not benchmarked to provide the necessary perspective, the results summary says “28% faster than the LuaJIT interpreter” before the scope of the research has been clarified, and only if you read through do you discover they did not make it generate JITs (yet). It seems everyone in this thread thinks there is now a faster Lua than LuaJIT. The author didn’t claim anything incorrect, but they did lead people to believe it.
> Lua is concise yet supports almost every language feature one can find in dynamic languages Having yet another terrible package manager? A non-existent ecosystem outside of checks notes game scripting and nginx? Reinvent-Everything where every developer everywhere has to reinvent everything poorly because the language is “concise” and “embeddable” and stuck in the 90s? Breaking changes between versions and interpre…
Fascinating! I wonder if there's a way to keep this 100% inside the C ecosystem, without having to reach for an LLVM dependency...
Not C, but could probably do it with Rust & Cranelift's IR instead of LLVM's. Not as heavy weight of a dependency as bringing in LLVM.
I just want to point out that LLVM is not a runtime dependency. It is only a build-time dependency if you want to build LJR from source. Once LJR is built, it is stand-alone and does not need LLVM at runtime.
Could this be used to build a better web assembly interpreter? wasm3 is the fastest Wasm interpreter I’ve found but this approach sounds very intriguing!
Wasm3 is already written in the tail-call style. Its IR consists of direct threaded code: a list of function pointers that each load and tail-call the next one. The only improvement on that I could see would be if there are spills because of the calling convention running out of registers.
Running out of registers in tail calls is one of the reasons I avoided that in doing Wizard's Wasm interpreter. One constraint there is that interpretation is done in-place (i.e. the original Wasm bytes). I wrote it in hand-rolled asm and it uses 9 architectural registers.
Relatedly, it’s worth noting that there are really good bindings between Lua and Haskell [ https://hslua.org/ ], used in e.g. Pandoc for plugins.
Haskell and Pandoc has too much dependency bloat. Just installing Pandoc on Arch requires like 200MB in dependencies, and over hundred haskell specific dependencies.
Well, firstly, I’d note that Haskell on Arch (including Pandoc) is broken, mostly due to the fact that Arch likes to dynamically link everything and Haskell doesn’t (see also https://www.reddit.com/r/haskell/comments/yuwcei). But yes, Haskell packages do often tend to have quite a lot of dependencies. To some extent this is because the base library is very much ‘batteries-excluded’, so dependencies are necessary for things like file management and byte strings and so on, but I do see quite a few dependencies which could be split up (e.g. ‘servant-server’, a big dependency used to create a webserver in exactly one module of pandoc).
One caveat to pay attention to... Ever since LuaJIT and PUC-Lua diverged, PUC lua has made some significant changes to the garbage collector. I wonder if that might affect the comparison vs LuaJIT for memory intensive benchmarks such as binarytrees and tablesort.
As detailed in the article, the garbage collectors for all interpreters tested were disabled for all benchmarks because this interpreter does not yet have garbage collection.