Live data from Hacker News

Building the fastest Lua interpreter automatically

sillycross.github.io

101–110 of 114 posts

Re: Building the fastest Lua interpreter automatically

#101

> Lua is concise yet supports almost every language feature one can find in dynamic languages Having yet another terrible package manager? A non-existent ecosystem outside of checks notes game scripting and nginx? Reinvent-Everything where every developer everywhere has to reinvent everything poorly because the language is “concise” and “embeddable” and stuck in the 90s? Breaking changes between versions and interpre…

What's wrong with luarocks?

Re: Building the fastest Lua interpreter automatically

#102

There's some prior art on DSLs for efficient interpreters, like Anton Ertl's vmgen. https://www.complang.tuwien.ac.at/anton/vmgen/

The main difference I see is this project seems to be using C++ (plus C macros?) directly instead of a DSL. And some llvm magic thrown in for good measure.

Re: Building the fastest Lua interpreter automatically

#103

Earlier quoted context omitted.

I tried exposing the "cc 10" and "cc 11" calling conventions to Clang, but was unsuccessful. With "cc 10" I got crashes at runtime I could not explain. With "cc 11" I got a compile-time error: > fatal error: error in backend: Can't generate HiPE prologue without runtime parameters If "cc 10" would work from Clang I'd be happy to use that. Though I think reverse_cc could potentially still offer benefits by ordering ar…

Adding calling conventions to clang is fairly straightforward (e.g. https://reviews.llvm.org/D125970 ) but there's a limited size structure in clang somewhere that makes doing so slightly contentious. Partly because they're shared across all architectures. At some point we'll run out of bits and have to do something invasive and/or slow to free up space.

Yes, I've already experimented with doing this, see: https://github.com/haberman/llvm-project/commit/e8d9c75bb35c...

But when I tried it on my actual code, the results weren't quite as good as I hoped, due to sub-optimal register allocation.

Re: Building the fastest Lua interpreter automatically

#104

There's some prior art on DSLs for efficient interpreters, like Anton Ertl's vmgen. https://www.complang.tuwien.ac.at/anton/vmgen/

The main difference I see is this project seems to be using C++ (plus C macros?) directly instead of a DSL. And some llvm magic thrown in for good measure.

Fair. I think of DSLs as a way of thinking about the program you want to write; if a scheme is realizable with few compromises in an existing language, it can still be considered "DSL style". SICP calls this style "stratified design".

Re: Building the fastest Lua interpreter automatically

#106
This seems like an awesome way of writing faster interpreters – i.e. not in assembly, but in C++ snippets you stitch together with a tool.

I did peek at the deegen tool a bit, and it seems quite large? https://github.com/luajit-remake/luajit-remake/tree/master/d...

I would be interested in an overview of all the analysis it has to do, which as I understand is basically “automated Mike Pall”

FWIW I think this is the hand-written equivalent with LuaJIT’s dynasm tool: https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/vm_x64.dasc (just under 5000 lines)

Also there are several of these files with no apparent sharing, as you would get with deegen:

https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/vm_x86.dasc

https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/vm_ppc.dasc

Re: Building the fastest Lua interpreter automatically

#107

I work on a game, that mostly uses lua as logic code, saddling on a c/c++ engine. One of the engine developers implemented lua jit years ago and found that for the actual performance, the interpreter/jit was neglibable, the most expensive thing being the actual switch between luacode and c-code, which is with a large API a constant ugly background performance loss. The lua code by itself did not ran long enough to ac…

Fatshark?

Re: Building the fastest Lua interpreter automatically

#108

Earlier quoted context omitted.

The main difference I see is this project seems to be using C++ (plus C macros?) directly instead of a DSL. And some llvm magic thrown in for good measure.

Fair. I think of DSLs as a way of thinking about the program you want to write; if a scheme is realizable with few compromises in an existing language, it can still be considered "DSL style". SICP calls this style "stratified design".

You can do all sorts of “stratified design” with C++ templates which I assume is underneath the covers of this system. boost::spirit is probably the most excessive use of this that I’ve seen so far.

Re: Building the fastest Lua interpreter automatically

#109
post #11

Nice to see Haskell make an appearance in an article about Lua and C/C++: > For example, at LLVM IR level, it is trivial to make a function use GHC calling convention (a convention with no callee-saved registers) This refers to the following section of [1] “cc 10” - GHC convention This calling convention has been implemented specifically for use by the Glasgow Haskell Compiler (GHC). It passes everything in registers…

I didn't realise LLVM supported such a wide variety of calling conventions. It would be cool if they supported OpenVMS/x86-64 calling convention–even not on OpenVMS. Why? Well, OpenVMS calling convention has this cool little feature which nobody else seems to have – the parameter count to a function is passed in a register. What this means–people always ask "how can a C variadic function know how many parameters it w…

Optimising variadic functions seems fairly low value but given they're usually a different calling convention anyway passing an extra integer is probably harmless.

Re: Building the fastest Lua interpreter automatically

#110

Earlier quoted context omitted.

I just want to point out that LLVM is not a runtime dependency. It is only a build-time dependency if you want to build LJR from source. Once LJR is built, it is stand-alone and does not need LLVM at runtime.

Do other JIT developers who would want to use Deegen need to pull in LLVM as a dependency still?

It should be a build time dependency. There's no JIT here, so there's no calling into LLVM's JIT, so I'd hope it's equivalent to using clang to create a native binary that implements the lua language.

Could probably retarget it to emit assembly and commit that for a loose interpretation of building from source.

Post reply on HN