Live data from Hacker News

Bringing GNU Emacs to native code [video]

toobnix.org

21–30 of 84 posts

Re: Bringing GNU Emacs to native code [video]

#21

Earlier quoted context omitted.

Emacs contains a quarter million lines of C. What do you do with all the elisp that calls into it? Maybe that could be turned into a "libemacs" and used from FFI.

Or maybe you rewrite it in Common Lisp. If you're going to take the Common Lisp route, I think it might be worth going all in and just re-implementing Emacs in Common Lisp.

That's already been done. More than once, in fact. The problem is that, although there are perfectly serviceable Emacsen written in Common Lisp, none of them are GNU Emacs.

For example, there's Hemlock from CMUCL, and its descendants built into Lispworks and Clozure Common Lisp.

Those implementations aren't likely to work well as a substitute for GNU Emacs. For one thing, there's a substantial ecosystem of software that depends on specific APIs and other characteristics of GNU Emacs. Writing code to bridge GNU Emacs APIs with those available in the Hemlock descendants would be a lot of work.

There are some other obstacles as well. CCL's implementation of Hemlock uses the Cocoa text architecture, so it's not portable to platforms other than macOS.

The Lispworks implementation is portable across Windows and numerous UNIXEN, but the Lispworks license will not allow delivery of a proper substitute for GNU Emacs (it forbids building an application that can be construed as a Lisp development system, and when I pressed them about exactly what limits that policy implies, they explicitly used Emacs as an example of an application that would be forbidden).

The original CMUCL Hemlock and its portable version are designed to work with CLX. It could probably be made to work in a modern X environment, but making it fit well into modern GUI environments and porting it to all the platforms GNU Emacs works on would be a huge amount of work.

I'm probably overlooking some other Emacsen, but I don't know of any off the top of my head for which the situation is any easier.

Re: Bringing GNU Emacs to native code [video]

#22
post #12

I tried this out on my system last night. Compile time was quite large (I would say about an hour and a half on my Ryzen 3600X.) I use the DOOM Emacs config, and was surprised to find most things working out of the box with the native compilation. I noticed no difference in startup time. The speed boost was surprisingly noticeable, however, when e.g. opening a buffer that causes a language server to start. Opening CC…

Similar story for me (though unfortunately on a few years old i7, so the compile time was well over 2h).

I haven't had any stability issues with it, either, it just works.

Re: Bringing GNU Emacs to native code [video]

#23

Earlier quoted context omitted.

Or maybe you rewrite it in Common Lisp. If you're going to take the Common Lisp route, I think it might be worth going all in and just re-implementing Emacs in Common Lisp.

That's already been done. More than once, in fact. The problem is that, although there are perfectly serviceable Emacsen written in Common Lisp, none of them are GNU Emacs. For example, there's Hemlock from CMUCL, and its descendants built into Lispworks and Clozure Common Lisp. Those implementations aren't likely to work well as a substitute for GNU Emacs. For one thing, there's a substantial ecosystem of software t…

It hasn't already been done, no. The Emacs variants you mentioned (that IMV should not be referred to as "Emacs") are from-scratch new implementations of just the concept behind Emacs, but not GNU Emacs itself. They are not compatible with GNU Emacs, they don't even support a small fraction of GNU Emacs features, and are thus doomed to obsolescence / non-existent market share.

So far, nobody -that I know of- has tried to move GNU Emacs to Common Lisp. GNU Emacs is a well-known, well-used platform and moving to Common Lisp could very well be worth the upfront costs.

Wishful thinking aside, the GNU Emacs development community has hunkered down behind Emacs Lisp, so improving it will bring immediate and tangible benefits to every GNU Emacs user. I love Common Lisp and do use SBCL for a lot of personal projects, but I also love GNU Emacs and I will take any improvements there that I can get. Watching Emacs Lisp become a more viable general purpose language (improved performance means an expanded set of problems that it can now address) is a great development. Stefan Monnier on emacs-devel:

  The main benefit of such a compiler is not to run existing Elisp
  code faster (99% of existing Elisp code runs fast enough that the
  user won't notice if it runs faster) but to make it practical to
  write other Elisp code which would otherwise be too slow. But for
  that to work well, you want the new compiler to be
  available "everywhere", rather than just on some platforms. So it
  will only start being useful when it works on GNU/Linux, macOS,
  Windows, ARM, RISC-V, x86, amd64, MIPS, younameit (AFAIK
  lbgccjit's CPU coverage is already good enough for that, so the
  main barrier here is the OS support), and when it's not just an
  option at compile-time but when it's included in all builds.

Re: Bringing GNU Emacs to native code [video]

#24
post #12

I tried this out on my system last night. Compile time was quite large (I would say about an hour and a half on my Ryzen 3600X.) I use the DOOM Emacs config, and was surprised to find most things working out of the box with the native compilation. I noticed no difference in startup time. The speed boost was surprisingly noticeable, however, when e.g. opening a buffer that causes a language server to start. Opening CC…

does it do anything to change/fix Emacs shitting itself on large buffers?

last time i gave it a try opening a large file would completely kill performance, and iirc in particular really long lines (ex. 1000+ chars) would make the thing chug even if the actual file wasn't super big or anything.

Re: Bringing GNU Emacs to native code [video]

#26
post #24
post #12

I tried this out on my system last night. Compile time was quite large (I would say about an hour and a half on my Ryzen 3600X.) I use the DOOM Emacs config, and was surprised to find most things working out of the box with the native compilation. I noticed no difference in startup time. The speed boost was surprisingly noticeable, however, when e.g. opening a buffer that causes a language server to start. Opening CC…

does it do anything to change/fix Emacs shitting itself on large buffers? last time i gave it a try opening a large file would completely kill performance, and iirc in particular really long lines (ex. 1000+ chars) would make the thing chug even if the actual file wasn't super big or anything.

Are you opening it in fundamental mode? Without actually doing profiling, I'd guess it's syntax highlighting that's the real issue there. I've opened 100+ MB files without an issue, but that was on a box with an absolute ton of RAM.

Re: Bringing GNU Emacs to native code [video]

#27

Earlier quoted context omitted.

Rewriting thousands of packages (some of which have had man-decades of work poured into them) would also be rather convoluted.

I wonder whether a modern modularized architecture might be the better way to handle this. Similar to neovim where you have a backend which communicates with a detatched frontend and other backends over some rpc-mechanism. Refactor the existing emacs to work headless and become the legacy-core, while also establishing channels to attatch any other core and frontend.

An important optimization discussed in the video is to optimize through call into C code (i.e elisp->C->elisp). adding RPC at the boundaries would go in the opposite direction.

Re: Bringing GNU Emacs to native code [video]

#28
post #24
post #12

I tried this out on my system last night. Compile time was quite large (I would say about an hour and a half on my Ryzen 3600X.) I use the DOOM Emacs config, and was surprised to find most things working out of the box with the native compilation. I noticed no difference in startup time. The speed boost was surprisingly noticeable, however, when e.g. opening a buffer that causes a language server to start. Opening CC…

does it do anything to change/fix Emacs shitting itself on large buffers? last time i gave it a try opening a large file would completely kill performance, and iirc in particular really long lines (ex. 1000+ chars) would make the thing chug even if the actual file wasn't super big or anything.

Large buffers are fine - long lines are not.

If you don't have long lines but you are experiencing slowdowns on large buffers, your mode is doing something. As an example, large XML files bring my Emacs to a crawl, but if I switch to a text mode it's fine.

For long lines, I believe the latest Emacs has a fix: https://www.reddit.com/r/emacs/comments/ccoksw/solong_mitiga...

Re: Bringing GNU Emacs to native code [video]

#29
post #8

Earlier quoted context omitted.

Rewriting thousands of packages (some of which have had man-decades of work poured into them) would also be rather convoluted.

You don't need to rewrite packages if you have a layer where Emacs Lisp is compiled/interpreted as Common Lisp. You can easily hack the readtable in Lisp, and rewrite Emacs Lisp sexps as Common Lisp sexps. Some interesting things to consider are file-local variables, buffer-local variables, dynamic-scope/lexical-scope, how to handle floating point (-0.0e+NaN in Emacs Lisp, custom floating point rouding modes/traps in…

>You can easily hack the readtable in Lisp

The funny thing about lisp is that all things are easy, but yet, other eco systems offer more finished libraries.

Re: Bringing GNU Emacs to native code [video]

#30
As a exclusive emacs user for the last 20 years, I'm quite excited; my main complain about emacs is its slow down with some more sophisticated packages. I wonder if it improves magit performance with large codebases.

Pigs fly just fine with with enough thrust.

Post reply on HN