A vision of a multi-threaded Emacs • Hacker News Board

Re: A vision of a multi-threaded Emacs

#41

Post by complex1314 » Mon, May 30, 2022, 7:16 PM UTC

dotancohen wrote: post #31

metroholografix wrote: post #10

Earlier quoted context omitted.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

I just got into emacs last year, for Org mode. Startup times drive me crazy. What would be the correct way to configure my .emacs file to e.g. check for package updates in a background process?

Use emacsclient and have a emacs server running at all times. Super fast startup times, you can even use emacsclient -nw (or make an alias 'e' for that) in the terminal for fast edits with instantaneous startup.

You could also make a script that starts emacs as a client only if a server already is running

  #!/bin/sh
  if [ "$#" -eq 0 ]
  then
    echo "Starting new Emacs process ..." >&2
    nohup emacs > /dev/null 2>&1 &
  elif emacsclient -n "$@" 2> /dev/null
  then
    echo "Opened $@ in Emacs server" >&2
  else
    echo "Opening $@ in a new Emacs process ..." >&2
    nohup emacs "$@" > /dev/null 2>&1 &
  fi

(Copied from an emacs starter kit but don't remember which...)

Re: A vision of a multi-threaded Emacs

#42

Post by zdragnar » Mon, May 30, 2022, 7:27 PM UTC

metroholografix wrote: post #10

Kaze404 wrote: post #6

I'd love to see this someday, as an Emacs user. One of my biggest frustrations with Emacs is how it can simply just hang while waiting for anything IO bound (like a response from LSP). It mostly only happens when opening a new file, while in VSCode for example the file is opened immediately and things like LSP and syntax highlighting are loaded in the background.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

> It is therefore up to you, to use the proper package

This is the single biggest reason I don't use emacs. I have yet to get a stable installation with the necessary behaviors and tools I need to make memorizing all the chords worth the effort.

I've tried spacemacs, doom and from scratch, and have never been satisfied or gotten to the point where coding is as productive as VScode for me.

Re: A vision of a multi-threaded Emacs

#43

Post by NeutralForest » Mon, May 30, 2022, 7:41 PM UTC

Kaze404 wrote: post #6

I'd love to see this someday, as an Emacs user. One of my biggest frustrations with Emacs is how it can simply just hang while waiting for anything IO bound (like a response from LSP). It mostly only happens when opening a new file, while in VSCode for example the file is opened immediately and things like LSP and syntax highlighting are loaded in the background.

My biggest pet-peeve is Org freezing when I'm exporting to PDF.

Re: A vision of a multi-threaded Emacs

#44

Post by funcDropShadow » Mon, May 30, 2022, 7:48 PM UTC

light_hue_1 wrote: post #24

jhgb wrote: post #21

Earlier quoted context omitted.

> and an incredibly slow language Languages are not slow or fast; their implementations are. There's absolutely nothing wrong with Emacs being programmed in Lisp and it sure as hell isn't "a poor architectural decision". If Emacs is slow because of its implementation of Emacs Lisp, it's time to fix its implementation of Emacs Lisp.

> There's absolutely nothing wrong with Emacs being programmed in Lisp and it sure as hell isn't "a poor architectural decision" I love Scheme and Lisp. I've probably written several million lines of code in both Scheme and Lisp and I've contributed code to several Lisp and Scheme compilers. But languages are absolutely slow or fast! There are fast Schemes and Lisps and there are slow ones. Emacs has probably the slo…

> But languages are absolutely slow or fast! There are fast Schemes and Lisps and there are slow ones.

You are contradicting yourself. If languages instead of implementations were slow or fast, then their would not be fast and slow Schemes and Lisps. Scheme would be either slow or fast.

Actually, I would go even further than saying only implementations are slow or fast, they are slow or fast for certain use cases.

> There is simply no efficient implementation of Elisp possible as it stands and as it is used today. Even with JIT compiling Elisp performance is simply wretched.

I agree in so far as there will --- probably --- never be an Elisp implementation which can run numeric intensive code as fast as the best Fortran or C compiler for this task. But it is certainly possible to improve the performance of ELisp even more than native-comp branch did. The question is whether there are people capable and willing to invest the time to develop that and whether spending that time on the language implementation is the most useful way to spend that time.

Re: A vision of a multi-threaded Emacs

#45

Post by yakubin » Mon, May 30, 2022, 7:50 PM UTC

metroholografix wrote: post #10

Kaze404 wrote: post #6

I'd love to see this someday, as an Emacs user. One of my biggest frustrations with Emacs is how it can simply just hang while waiting for anything IO bound (like a response from LSP). It mostly only happens when opening a new file, while in VSCode for example the file is opened immediately and things like LSP and syntax highlighting are loaded in the background.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

For me the most frequent source of freezes was Tramp. It led me to ditch Emacs for VSCode (with the remote editing extension) at least for my dayjob (I still use Emacs for my own projects). Tramp is an official built-in package, so it's not as if I downloaded a random package from GitHub.

Re: A vision of a multi-threaded Emacs

#46

Post by taeric » Mon, May 30, 2022, 7:55 PM UTC

metroholografix wrote: post #10

Kaze404 wrote: post #6

I'd love to see this someday, as an Emacs user. One of my biggest frustrations with Emacs is how it can simply just hang while waiting for anything IO bound (like a response from LSP). It mostly only happens when opening a new file, while in VSCode for example the file is opened immediately and things like LSP and syntax highlighting are loaded in the background.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

Glad to see this as the top response here. I was rushing to click reply so that I could point out that async has been a thing for Emacs for a long long time. Compilation mode is my goto example for how that can work.

Re: A vision of a multi-threaded Emacs

#47

Post by taeric » Mon, May 30, 2022, 7:57 PM UTC

TacticalCoder wrote: post #40

Kaze404 wrote: post #6

I'd love to see this someday, as an Emacs user. One of my biggest frustrations with Emacs is how it can simply just hang while waiting for anything IO bound (like a response from LSP). It mostly only happens when opening a new file, while in VSCode for example the file is opened immediately and things like LSP and syntax highlighting are loaded in the background.

> ... while in VSCode for example the file is opened immediately and things like LSP and syntax highlighting are loaded in the background. I'm not saying you're not experiencing what you're describing but I find that weird because I use Emacs with LSP and the system is responsive while LSP loads up. And I'm using lsp-mode, not Eglot (I should really try Eglot). I mostly use it for Clojure: I'm using both Cider and ls…

I have seen some folks that don't realize they are bypassing the native-json parser by having debug logging turned on. So, could be that?

Re: A vision of a multi-threaded Emacs

#48

Post by taeric » Mon, May 30, 2022, 7:59 PM UTC

CJefferson wrote: post #5

Note: Not at all an emacs expert, but know some stuff about trying to parallelise a long-standing language ( GAP -- www.gap-system.org , 30 year old maths language). I'd strongly recommend something like Javascript's "web workers", where each thread lives in an almost entirely independent memory space, with sharing limited to explicitly named buffers. The problem with traditional "everything shared by default" thread…

I'd argue that emacs already has that with process buffers using sentinels. :D

By and large, if you don't write blocking code, it doesn't block. Big caveat here is the TRAMP logic to establish connections can block. Not sure why those weren't done in a more async way.

Re: A vision of a multi-threaded Emacs

#49

Post by funcDropShadow » Mon, May 30, 2022, 8:06 PM UTC

CJefferson wrote: post #5

Note: Not at all an emacs expert, but know some stuff about trying to parallelise a long-standing language ( GAP -- www.gap-system.org , 30 year old maths language). I'd strongly recommend something like Javascript's "web workers", where each thread lives in an almost entirely independent memory space, with sharing limited to explicitly named buffers. The problem with traditional "everything shared by default" thread…

In principle I agree whole heartedly. When introducing parallelism and or concurrency into an old language, the "web workers" aproach, or Erlangs actors, to name a decades older reference to a similar idea, should always be considered. The problem with emacs in particular is the use of the buffer as the most versatile data structure. Lot's of data that could live in some data structure lives in some overlay over some part of the buffer. And emacs is built on dynamic variables. Others in this thread have argued to "just refactor" them to lexically scoped variables. But that would decimate the largest vector to adapt and modify the system.

Parallelizing Emacs needs to be a long-term endaveour, as the maintainers of Emacs already said years ago. I think Emacs could learn something from Clojure's story for parallelism. Make data structures immutable by default, and reference them from as few global variables as possible. The "make data structures immutable by default" ship has already sailed, but introducing additional immutable list/vector/map/set data structures with first class syntax support would allow newly written code to prepare for a more parallel future. Such immutable data structures could then be shared between different "ELisp web workers". The "as few global variables story" ship has already sailed, but again ELisp authors would start to consider reducing the number of global variables if it would enable multi-threading.

Re: A vision of a multi-threaded Emacs

#50

Post by natrys » Mon, May 30, 2022, 8:18 PM UTC

light_hue_1 wrote: post #20

metroholografix wrote: post #10

Earlier quoted context omitted.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

> I can definitely say that if you're experiencing I/O hangs, it's either misconfiguration or bad code in a package that you loaded. As an emacs user and package developer of 20 years I can definitely say you are totally wrong. This is the kind of dismissive, unhelpful, and frankly extremely discouraging attitude that is slowly killing emacs. Instead of taking responsibility for the fact that emacs is incredibly slow…

The presupposition that anything is killing Emacs - much less this - requires concrete proof. Because if anything, in my (likewise subjective) experience it's the complete opposite. I see more forum activity from newcomers and more high quality packages emerging each year than ever before.

> like an insistence to not using fast native libraries

How? Even disregarding graphics and font stack which almost entirely leverages native libraries, my Emacs build links against gmp, libpng, librsvg, zlib, alsa-lib, libxml2 and so on. 27.1 introduced jansson for native json parsing over pure elisp, from 28.1 we can basically AoT compile elisp to native code using libgccjit, master branch (29) even bundles sqlite these days. That is not to mention module system had been stable for a while, and lot of packages already use this to offload computation heavy stuff to native code.