Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bringing GNU Emacs to native code [video]

toobnix.org

Post reply on HN
Print view View original

51–60 of 84 posts

Re: Bringing GNU Emacs to native code [video]

#51

Post by hvis »

bjoli wrote: post #19
ufo wrote: post #17

Earlier quoted context omitted.

I would expect that an ELisp interpreter written in Common Lisp would likely be slower than the existing ELisp interpreter written in C. As for compilation to Common Lisp, I don't know how feasible it would be. Apparently they tried something similar with Guile and it ran into problems.

Guile (the VM) runs elisp just fine. There has been zero optimization work done though so it is quite slow. The reason they did not go with guile was more political than anything else. It was also understandable from their point of view (and I say this as a guile weenie)

It was somewhat political, but it was more of an issue of manpower. Also, Guile is basically maintained by one developer (as far as I've heard). It's pretty dangerous for Emacs to rely on a project like that.

Re: Bringing GNU Emacs to native code [video]

#52

Post by __s »

philsnow wrote: post #43
harrygeez wrote: post #25

Just when I thought I know enough compsci to understand everything on a sufficiently high level, this guy totally lost me starting from LIMPLE

Compilers are "just" a series of transformations / translations from higher-level code to lower-level code. The top is code like C, python, elisp, whatever, and the bottom is machine code for amd64, arm7, whatever. All the in-between code is in some intermediate representation (IR). Each successive step takes care of different optimizations, modifying the code as it goes down. At the last step, he converts LIMPLE to…

Also even if you do "one step" you'll want to have an intermediate data format which is SSA for the more efficient optimizations

Re: Bringing GNU Emacs to native code [video]

#53

Post by pulisse »

morelisp wrote: post #41
aasasd wrote: post #33

Earlier quoted context omitted.

> in particular really long lines (ex. 1000+ chars) would make the thing chug My uninformed guess is, this is bottlenecked by an inefficient algorithm that accesses the memory too much, i.e. it's nonlinear with respect to the number of characters. I doubt that tweaking the overall speed would fix it.

My experience is it's generally either - inefficient font-lock regexes, which are rarely benchmarked / optimized since most files don't have long lines and because font-lock behavior is quite complicated to begin with; - inefficient thing-at-point implementations / use, e.g. an O(n*n) thing-at-point called at various points by an O(n) function; - modes using font-lock regexps where they really just need "fixed" style…

Emacs dev here. The primary cause is something else.

Emacs's inefficiency in handling files with long lines is due to two factors: (a) The primitive unit of work for the display engine (the code that determines how to combine text, font metrics, syntax highlighting, inline image display, etc.) is a line (a newline-delimited span of characters). (b) The redisplay routine is called very frequently—not just when the screen is repainted, but pretty much any time the screen location of a buffer element need to be calculated, and so, e.g., during navigation. So Emacs is constantly, under the hood, going back to the previous newline and re-calculating how the buffer contents from that point forward should be rendered.

Re: Bringing GNU Emacs to native code [video]

#54

Post by na85 »

s-km wrote: post #24
poidos wrote: post #12

I tried this out on my system last night. Compile time was quite large (I would say about an hour and a half on my Ryzen 3600X.) I use the DOOM Emacs config, and was surprised to find most things working out of the box with the native compilation. I noticed no difference in startup time. The speed boost was surprisingly noticeable, however, when e.g. opening a buffer that causes a language server to start. Opening CC…

does it do anything to change/fix Emacs shitting itself on large buffers? last time i gave it a try opening a large file would completely kill performance, and iirc in particular really long lines (ex. 1000+ chars) would make the thing chug even if the actual file wasn't super big or anything.

I routinely edit C# files in the 5k to 15k LOC range, and emacs works well for me. Occasionally the "smooth scrolling" feature stutters on large files but otherwise it's perfectly usable.

Re: Bringing GNU Emacs to native code [video]

#55

Post by tgbugs »

I have a Gentoo ebuild [0] almost working for this (I think I'm missing the step where the eln files are loaded before dumping the base image). The compile times are ... substantial. However they don't affect the development process for new code since the interpreter is always there, and I am excited to see what performance gains we will see.

From an engineering perspective this is a excellent example of a direct path from interpreted to compiled code. The trade-offs are clear (heck, they are number 0-3), and while there is complexity, all the engineering time has been effectively concentrated inside a single project, rather than forced upon tens of thousands of maintainers and users. Bravo. I wonder what other bytecode interpreters could benefit from this toolchain. Compile times from qlop.

On a Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz Without comp `2020-05-03T15:33:27 >>> app-editors/emacs: 6′40″` With compile `2020-05-03T18:35:43 >>> app-editors/emacs: 2:11:45`

0. https://github.com/tgbugs/tgbugs-overlay/blob/master/app-edi...

Re: Bringing GNU Emacs to native code [video]

#56

Post by fiddlerwoaroof »

mikelevins wrote: post #21
wtetzner wrote: post #14

Earlier quoted context omitted.

Or maybe you rewrite it in Common Lisp. If you're going to take the Common Lisp route, I think it might be worth going all in and just re-implementing Emacs in Common Lisp.

That's already been done. More than once, in fact. The problem is that, although there are perfectly serviceable Emacsen written in Common Lisp, none of them are GNU Emacs. For example, there's Hemlock from CMUCL, and its descendants built into Lispworks and Clozure Common Lisp. Those implementations aren't likely to work well as a substitute for GNU Emacs. For one thing, there's a substantial ecosystem of software t…

Robert Strandh is working on an Common Lisp emacs based on McClim but, McClim basically forces you to use linux.

Re: Bringing GNU Emacs to native code [video]

#57

Post by tgbugs »

kazinator wrote: post #11
fmakunbound wrote: post #2

This seems convoluted compared to say moving the Lisp implementation from Emacs Lisp to Common Lisp, of which several native code compiling implementations exists.

It brings GCC into the address space via libgccjit. GCC is not robust enough to be integrated into applications that stay running, and be repeatedly invoked.

I don't think that is how libgccjit is being used in this project. Emacs is not being linked with libgccjit. libgccjit is only being use for AOT compilation.

Re: Bringing GNU Emacs to native code [video]

#59

Post by fiddlerwoaroof »

fiddlerwoaroof wrote: post #56
mikelevins wrote: post #21

Earlier quoted context omitted.

That's already been done. More than once, in fact. The problem is that, although there are perfectly serviceable Emacsen written in Common Lisp, none of them are GNU Emacs. For example, there's Hemlock from CMUCL, and its descendants built into Lispworks and Clozure Common Lisp. Those implementations aren't likely to work well as a substitute for GNU Emacs. For one thing, there's a substantial ecosystem of software t…

Robert Strandh is working on an Common Lisp emacs based on McClim but, McClim basically forces you to use linux.

There's also lem, which is more portable: https://github.com/cxxxr/lem

Re: Bringing GNU Emacs to native code [video]

#60

Post by morelisp »

pulisse wrote: post #53
morelisp wrote: post #41

Earlier quoted context omitted.

My experience is it's generally either - inefficient font-lock regexes, which are rarely benchmarked / optimized since most files don't have long lines and because font-lock behavior is quite complicated to begin with; - inefficient thing-at-point implementations / use, e.g. an O(n*n) thing-at-point called at various points by an O(n) function; - modes using font-lock regexps where they really just need "fixed" style…

Emacs dev here. The primary cause is something else. Emacs's inefficiency in handling files with long lines is due to two factors: (a) The primitive unit of work for the display engine (the code that determines how to combine text, font metrics, syntax highlighting, inline image display, etc.) is a line (a newline-delimited span of characters). (b) The redisplay routine is called very frequently—not just when the scr…

I’m not an Emacs dev so I’ll concede your expertise here - but if it is fundamental to the core navigation and redisplay, why does “stepping down” modes help so much? Fundamental and text mode certainly aren’t great with long lines, but are usually orders of magnitude better (eg a few seconds per command instead of 10s of seconds).

(I have not tried so-long-mode and am mostly on Emacs 26 with some 25.)

Post reply on HN