Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bringing GNU Emacs to native code [video]

toobnix.org

Post reply on HN
Print view View original

31–40 of 84 posts

Re: Bringing GNU Emacs to native code [video]

#31

Post by rml »

s-km wrote: post #24
poidos wrote: post #12

I tried this out on my system last night. Compile time was quite large (I would say about an hour and a half on my Ryzen 3600X.) I use the DOOM Emacs config, and was surprised to find most things working out of the box with the native compilation. I noticed no difference in startup time. The speed boost was surprisingly noticeable, however, when e.g. opening a buffer that causes a language server to start. Opening CC…

does it do anything to change/fix Emacs shitting itself on large buffers? last time i gave it a try opening a large file would completely kill performance, and iirc in particular really long lines (ex. 1000+ chars) would make the thing chug even if the actual file wasn't super big or anything.

I've had good luck on big (multi-GB) SQL dumps, which often have very long lines, with

https://github.com/m00natic/vlfi

Re: Bringing GNU Emacs to native code [video]

#32

Post by metroholografix »

gpderetta wrote: post #30

As a exclusive emacs user for the last 20 years, I'm quite excited; my main complain about emacs is its slow down with some more sophisticated packages. I wonder if it improves magit performance with large codebases. Pigs fly just fine with with enough thrust.

One reason magit is slow (that last I checked still hasn't been fixed) is that it spawns a large number of processes calling out to git for each operation. Most of these are redundant and could/should go away with either a redesign or a working caching scheme. This issue is more than obvious on platforms (e.g. some macOS versions) where fork/vfork are not as fast as one would expect. I like the paradigm behind magit, but the implementation leaves a lot to be desired. So the pig in this case is definitely not Emacs.

Example:

  # Doing a simple magit refresh results in the following processes being spawned (tracked with dtrace)
  # Notice how many calls are completely redundant

  2020 May  4 12:46:22 11819  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11820  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11821  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11822  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11823  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11824  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11825  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11826  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11827  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11828  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11829  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11830  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11831  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11832  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11833  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11834  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11835  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11836  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11837  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11838  64b  /opt/local/bin/git --no-pager -c core.preloadindex=true -c log.showSignature=false 
  2020 May  4 12:46:22 11839  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11840  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c 
  2020 May  4 12:46:22 11841  64b  /opt/local/bin/git --no-pager -c core.preloadindex=true -c log.showSignature=false 
  2020 May  4 12:46:22 11842  64b  /opt/local/bin/git --no-pager -c core.preloadindex=true -c log.showSignature=false 
  2020 May  4 12:46:22 11843  64b  /opt/local/bin/git --no-pager -c core.preloadindex=true -c log.showSignature=false 
  2020 May  4 12:46:22 11844  64b  /opt/local/bin/git --no-pager -c core.preloadindex=true -c log.showSignature=false 
  2020 May  4 12:46:22 11845  64b  /opt/local/bin/git --no-pager -c core.preloadindex=true -c log.showSignature=false 
  2020 May  4 12:46:22 11846  64b  /opt/local/bin/git --no-pager --literal-pathspecs -c core.preloadindex=true -c

Re: Bringing GNU Emacs to native code [video]

#33

Post by aasasd »

s-km wrote: post #24
poidos wrote: post #12

I tried this out on my system last night. Compile time was quite large (I would say about an hour and a half on my Ryzen 3600X.) I use the DOOM Emacs config, and was surprised to find most things working out of the box with the native compilation. I noticed no difference in startup time. The speed boost was surprisingly noticeable, however, when e.g. opening a buffer that causes a language server to start. Opening CC…

does it do anything to change/fix Emacs shitting itself on large buffers? last time i gave it a try opening a large file would completely kill performance, and iirc in particular really long lines (ex. 1000+ chars) would make the thing chug even if the actual file wasn't super big or anything.

> in particular really long lines (ex. 1000+ chars) would make the thing chug

My uninformed guess is, this is bottlenecked by an inefficient algorithm that accesses the memory too much, i.e. it's nonlinear with respect to the number of characters. I doubt that tweaking the overall speed would fix it.

Re: Bringing GNU Emacs to native code [video]

#34

Post by mikelevins »

metroholografix wrote: post #23
mikelevins wrote: post #21

Earlier quoted context omitted.

That's already been done. More than once, in fact. The problem is that, although there are perfectly serviceable Emacsen written in Common Lisp, none of them are GNU Emacs. For example, there's Hemlock from CMUCL, and its descendants built into Lispworks and Clozure Common Lisp. Those implementations aren't likely to work well as a substitute for GNU Emacs. For one thing, there's a substantial ecosystem of software t…

It hasn't already been done, no. The Emacs variants you mentioned (that IMV should not be referred to as "Emacs") are from-scratch new implementations of just the concept behind Emacs, but not GNU Emacs itself. They are not compatible with GNU Emacs, they don't even support a small fraction of GNU Emacs features, and are thus doomed to obsolescence / non-existent market share. So far, nobody -that I know of- has trie…

You appear to consider "Emacs" and "GNU Emacs" synonymous. I explicitly do not.

Hemlock is not a reimplementation of GNU Emacs; it's derived from earlier Emacsen, just as GNU Emacs is. Hemlock in particular is based on ZWEI, and was in version 0.99 around the time that work started on the GNU Emacs project. It's a slightly older sibling of GNU Emacs, rather than a descendant of it.

GNU Emacs has undoubtedly become the de facto standard Emacs, but I don't see why that disqualifies other branches of the Emacs family tree from using the name that all of them inherit from the venerable TECO implementation.

My point was that Emacsen have already been built in Common Lisp, but that they do not serve as substitutes for GNU Emacs--a point with which you appear to agree.

Maybe we even agree on the reason why: because no Emacs implementation can substitute for GNU Emacs without supporting its ecosystem. The amount of work needed to do that would be enormous, whether the proposed substitute is written in Common Lisp or in something else.

Re: Bringing GNU Emacs to native code [video]

#35

Post by throwaway_pdp09 »

xxpor wrote: post #26
s-km wrote: post #24

Earlier quoted context omitted.

does it do anything to change/fix Emacs shitting itself on large buffers? last time i gave it a try opening a large file would completely kill performance, and iirc in particular really long lines (ex. 1000+ chars) would make the thing chug even if the actual file wasn't super big or anything.

Are you opening it in fundamental mode? Without actually doing profiling, I'd guess it's syntax highlighting that's the real issue there. I've opened 100+ MB files without an issue, but that was on a box with an absolute ton of RAM.

Nah. IME it really is just line length.

Re: Bringing GNU Emacs to native code [video]

#36

Post by yepguy »

s-km wrote: post #24
poidos wrote: post #12

I tried this out on my system last night. Compile time was quite large (I would say about an hour and a half on my Ryzen 3600X.) I use the DOOM Emacs config, and was surprised to find most things working out of the box with the native compilation. I noticed no difference in startup time. The speed boost was surprisingly noticeable, however, when e.g. opening a buffer that causes a language server to start. Opening CC…

does it do anything to change/fix Emacs shitting itself on large buffers? last time i gave it a try opening a large file would completely kill performance, and iirc in particular really long lines (ex. 1000+ chars) would make the thing chug even if the actual file wasn't super big or anything.

I think I remember Emacs 27 will include some improvements in that regard, even without native code compilation.

Re: Bringing GNU Emacs to native code [video]

#37

Post by reikonomusha »

toohotatopic wrote: post #29
junke wrote: post #8

Earlier quoted context omitted.

You don't need to rewrite packages if you have a layer where Emacs Lisp is compiled/interpreted as Common Lisp. You can easily hack the readtable in Lisp, and rewrite Emacs Lisp sexps as Common Lisp sexps. Some interesting things to consider are file-local variables, buffer-local variables, dynamic-scope/lexical-scope, how to handle floating point (-0.0e+NaN in Emacs Lisp, custom floating point rouding modes/traps in…

>You can easily hack the readtable in Lisp The funny thing about lisp is that all things are easy, but yet, other eco systems offer more finished libraries.

Other ecosystems also become obsolete or dead within years counted on one hand. That’s perfect for today’s culture of disposable code though. Lisp isn’t good for that.

Re: Bringing GNU Emacs to native code [video]

#38

Post by travisjeffery »

metroholografix wrote: post #32
gpderetta wrote: post #30

As a exclusive emacs user for the last 20 years, I'm quite excited; my main complain about emacs is its slow down with some more sophisticated packages. I wonder if it improves magit performance with large codebases. Pigs fly just fine with with enough thrust.

One reason magit is slow (that last I checked still hasn't been fixed) is that it spawns a large number of processes calling out to git for each operation. Most of these are redundant and could/should go away with either a redesign or a working caching scheme. This issue is more than obvious on platforms (e.g. some macOS versions) where fork/vfork are not as fast as one would expect. I like the paradigm behind magit,…

Magit is working on switching to libgit2 https://github.com/magit/magit/issues/2959 to fix this.

Re: Bringing GNU Emacs to native code [video]

#39

Post by gpderetta »

metroholografix wrote: post #32
gpderetta wrote: post #30

As a exclusive emacs user for the last 20 years, I'm quite excited; my main complain about emacs is its slow down with some more sophisticated packages. I wonder if it improves magit performance with large codebases. Pigs fly just fine with with enough thrust.

One reason magit is slow (that last I checked still hasn't been fixed) is that it spawns a large number of processes calling out to git for each operation. Most of these are redundant and could/should go away with either a redesign or a working caching scheme. This issue is more than obvious on platforms (e.g. some macOS versions) where fork/vfork are not as fast as one would expect. I like the paradigm behind magit,…

that's probably one reason, but as far as I can tell, a large cost is formatting an colouring the master diff buffer when there are a lot of changes. When doing very large merges, it takes forever to refresh the buffer but aborting whatever magit is doing will leave an uncoloured but perfectly usable buffer.

Re: Bringing GNU Emacs to native code [video]

#40

Post by metroholografix »

travisjeffery wrote: post #38
metroholografix wrote: post #32

Earlier quoted context omitted.

One reason magit is slow (that last I checked still hasn't been fixed) is that it spawns a large number of processes calling out to git for each operation. Most of these are redundant and could/should go away with either a redesign or a working caching scheme. This issue is more than obvious on platforms (e.g. some macOS versions) where fork/vfork are not as fast as one would expect. I like the paradigm behind magit,…

Magit is working on switching to libgit2 https://github.com/magit/magit/issues/2959 to fix this.

This issue has been open since 2017 and it's not panacea since it comes with its own -substantial- tradeoffs which is why as far as I know, magit is not "switching to libgit2" but planning to offer libgit2 as an additional option. Also for such an important feature, rather than going with FFI, it would be much better if Emacs linked with libgit2 (similarly to the improved JSON support in 27). That way the core Emacs development team would review all code and have a say in its design. That also implies magit being bundled with Emacs (or at the very least available in ELPA) which may not be realistic in terms of copyright assignment / code quality concerns.

Performance issues aside, a pure Lisp magit is the simplest and more universal setup which is why an attempt to fix the process spawning problems there should be made.

Post reply on HN