Live data from Hacker News

A vision of a multi-threaded Emacs

coredumped.dev

21–30 of 136 posts

Re: A vision of a multi-threaded Emacs

#21

Earlier quoted context omitted.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

> I can definitely say that if you're experiencing I/O hangs, it's either misconfiguration or bad code in a package that you loaded. As an emacs user and package developer of 20 years I can definitely say you are totally wrong. This is the kind of dismissive, unhelpful, and frankly extremely discouraging attitude that is slowly killing emacs. Instead of taking responsibility for the fact that emacs is incredibly slow…

> and an incredibly slow language

Languages are not slow or fast; their implementations are. There's absolutely nothing wrong with Emacs being programmed in Lisp and it sure as hell isn't "a poor architectural decision". If Emacs is slow because of its implementation of Emacs Lisp, it's time to fix its implementation of Emacs Lisp.

Re: A vision of a multi-threaded Emacs

#22
post #4
post #3

Clojure is memory safe and no data races possible?

Memory safe, perhaps, but data races are definitely possible. The main issue is that their STM system doesn't scale very well. It is good enough if you want to keep a simple CRUD GUI in sync but don't expect to be building high performance web servers anytime soon with Clojure atomics. You can't just create and throw those around willy-nilly like Goroutines. Clojure people doesn't like talking about the language's we…

STM, atomics and channels are all different things that can be use to build different kinds of systems, so I think it's good to disambiguate them when talking about concurrency problems and their tradeoffs.

Clojure' STM is basically unused. It's a suite of functions and a container type `ref` that allows you to do transactions in memory and keep everything consistent within the view of a transaction. The overhead required to do the book keeping for these transactions is often not worth it for most use cases in practice.

Clojure also, as an alternative, gives you `atom` which is a type that implements a simple thread safe compare-and-swap. This ends up being sufficient for many cases of sharing state between threads. When combined with Clojure's immutable data structures, one can often use them as a simple in memory database.

Those two things are about sharing memory, but has nothing to do with what the article talks about: green threads.

clojure.core.async, as the article succinctly explains, allows you to create "goroutines" or green threads which are then multiplexed across some number of OS threads. This can allow you to create many more threads for computation than you have OS threads, and combined with thread locals and atoms can allow one to build large scale systems.

Clojure's STM and atomics are sort of a non-sequitur from what the article is talking about

Now as for downsides of Clojure, there are plenty of gotchas in practice that can lead people to issues with the three things above:

* STM is slow and rarely useful compared to regular atomics

* atoms CAS assumes you are using immutable data structures

* core.async has a lot of sharp edges, including lack of first-class error handling and the fact that doing synchronous I/O on a go routine can end up starving the thread pool they are running on

I've never built a high performance web server, but I imagine it's possible with Clojure; it would probably require you to probably eschew immutable data structures within the plumbing (for performance) and use an async I/O framework along with core.async to get competitive with a language like go. For shared memory, you probably would end up eschewing compare-and-swap for something more bare bones like locks, which you have access to via Java interop. At that point, you may wonder why you even chose Clojure in the first place instead of using Java directly. :D

Project Loom really only helps solve the problem that core.async is trying to solve, and applications would still make use of atomics like `atom` and basic Java atomic values. My hope is that it makes it easier to do I/O, since it sounds like along with loom fibers the project is adding first class support for suspending and resuming on fundamental I/O operations. Time will tell.

Re: A vision of a multi-threaded Emacs

#23
post #6

I'd love to see this someday, as an Emacs user. One of my biggest frustrations with Emacs is how it can simply just hang while waiting for anything IO bound (like a response from LSP). It mostly only happens when opening a new file, while in VSCode for example the file is opened immediately and things like LSP and syntax highlighting are loaded in the background.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

I've seen you post this criticism before, but the line "volunteers on GitHub" is pure FUD. Is the Eglot guy paid for writing Eglot, or he hosts somewhere else, so it's better?

"The code is nowhere near as good" according to you? Give a concrete example of where the LSP code has real deficiencies; Eglot isn't as fully featured, is more opinionated, and its stans are more annoying.

Sounds to me like the LSP project is helping new hackers learn Emacs, and Eglot is one bus accident from being unmaintained, frankly.

Re: A vision of a multi-threaded Emacs

#24
post #21

Earlier quoted context omitted.

> I can definitely say that if you're experiencing I/O hangs, it's either misconfiguration or bad code in a package that you loaded. As an emacs user and package developer of 20 years I can definitely say you are totally wrong. This is the kind of dismissive, unhelpful, and frankly extremely discouraging attitude that is slowly killing emacs. Instead of taking responsibility for the fact that emacs is incredibly slow…

> and an incredibly slow language Languages are not slow or fast; their implementations are. There's absolutely nothing wrong with Emacs being programmed in Lisp and it sure as hell isn't "a poor architectural decision". If Emacs is slow because of its implementation of Emacs Lisp, it's time to fix its implementation of Emacs Lisp.

> There's absolutely nothing wrong with Emacs being programmed in Lisp and it sure as hell isn't "a poor architectural decision"

I love Scheme and Lisp. I've probably written several million lines of code in both Scheme and Lisp and I've contributed code to several Lisp and Scheme compilers.

But languages are absolutely slow or fast! There are fast Schemes and Lisps and there are slow ones. Emacs has probably the slowest one by far. For example, Emacs still relies on dynamic binding after all of this time. That's a performance disaster.

There is simply no efficient implementation of Elisp possible as it stands and as it is used today. Even with JIT compiling Elisp performance is simply wretched.

Re: A vision of a multi-threaded Emacs

#25

Earlier quoted context omitted.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

While I appreciate the comments on Eglot vs LSP, the number one package that hangs my system is Gnus. It's still my favourite e-mail client, but I'm cut off from coding for twenty seconds every time I check my e-mail. Are we really ready to declare that Lars Magne Ingebrigtsen is a substandard Emacs Lisp coder?

Gnus has been around since the mid 90s (and predates the addition of asynchronous url-retrieve api to emacs). It is unfair to compare it with packages that written in the last couple of years.

I don't use Gnus so I have never bothered looking at its internals, but if it is locking your emacs for twenty seconds checking email, it is probably not doing much asynchronously. It is notoriously hard to convert existing synchronous code to asynchronous without using threads. Promises/futures/continuations/whatever-you-want-to-call-them are a pretty all in affair. Not having the time investment that would take to convert a 100k line codebase to async does not make Lars Magne Ingebrigtsen a substandard Emacs Lisp coder.

Re: A vision of a multi-threaded Emacs

#26
One big challenge for multi-threaded Emacs is that there are many elisp libraries and common patterns around buffer modification that make implicit assumptions about sequential and exclusive execution.

The various race conditions that would appear in a multithreaded context would be a nightmare to debug because they would likely only occur sporadically and would be hard to reproduce. I would love to be wrong about this, but I see it as a major obstacle that will likely need to be sandboxed (as was done with lexical scoping).

For an example of how hard these kinds of issues could be to debug look no further than the incredible difficult that the undo-tree author had in debugging a race condition with the garbage collector. It took years.

In the context of this piece, having a buffer mutex makes certain use cases for multi-threading impossible, for example dividing the buffer up into chunk equal to the number of cores and running a regexp over each chunk.

Re: A vision of a multi-threaded Emacs

#27

Earlier quoted context omitted.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

> I can definitely say that if you're experiencing I/O hangs, it's either misconfiguration or bad code in a package that you loaded. As an emacs user and package developer of 20 years I can definitely say you are totally wrong. This is the kind of dismissive, unhelpful, and frankly extremely discouraging attitude that is slowly killing emacs. Instead of taking responsibility for the fact that emacs is incredibly slow…

> This is the kind of dismissive, unhelpful, and frankly extremely discouraging attitude that is slowly killing emacs.

Emacs seems to be doing just fine if not thriving.

> Instead of taking responsibility for the fact that emacs is incredibly slow because of poor architectural decisions (like an insistence to not using fast native libraries and an incredibly slow language)

For me, Emacs is both incredibly fast and incredibly stable. For an "Emacs user and package developer of 20 years", you seem either too quick to jump to conclusions or are still barely scratching the surface of what's possible which makes me question the depth of your expertise and/or how efficiently your time with Emacs was invested.

Re: A vision of a multi-threaded Emacs

#28

Earlier quoted context omitted.

While I appreciate the comments on Eglot vs LSP, the number one package that hangs my system is Gnus. It's still my favourite e-mail client, but I'm cut off from coding for twenty seconds every time I check my e-mail. Are we really ready to declare that Lars Magne Ingebrigtsen is a substandard Emacs Lisp coder?

Gnus has been around since the mid 90s (and predates the addition of asynchronous url-retrieve api to emacs). It is unfair to compare it with packages that written in the last couple of years. I don't use Gnus so I have never bothered looking at its internals, but if it is locking your emacs for twenty seconds checking email, it is probably not doing much asynchronously. It is notoriously hard to convert existing syn…

This. Gnus is a behemoth, has been around for decades and is also the result of many contributions from many different people. It will take time for it to evolve (it's slowly happening).

Lars is amazing by the way, his relentless Emacs work and prolific bug killing in the last few years should be praised to high heavens.

Re: A vision of a multi-threaded Emacs

#29
post #21

Earlier quoted context omitted.

> and an incredibly slow language Languages are not slow or fast; their implementations are. There's absolutely nothing wrong with Emacs being programmed in Lisp and it sure as hell isn't "a poor architectural decision". If Emacs is slow because of its implementation of Emacs Lisp, it's time to fix its implementation of Emacs Lisp.

> There's absolutely nothing wrong with Emacs being programmed in Lisp and it sure as hell isn't "a poor architectural decision" I love Scheme and Lisp. I've probably written several million lines of code in both Scheme and Lisp and I've contributed code to several Lisp and Scheme compilers. But languages are absolutely slow or fast! There are fast Schemes and Lisps and there are slow ones. Emacs has probably the slo…

Then the codebase needs to fully (or as much as feasible) transition to the use of lexical bindings. It's not like we don't know if it will work; Common Lisp supports dynamic variables but you don't pay for them unless you actually use them, and good implementations of CL are fast enough. So what exactly is it that prevents the Emacs Lisp code base from becoming fast enough by abandoning dynamic variables wherever it can? By all rights it should not be any slower than Common Lisp at that point, unless some other obstacle has somehow eluded my attention.

Re: A vision of a multi-threaded Emacs

#30
post #15

Earlier quoted context omitted.

That multi-threading will solve this issue, is a common misconception. Emacs already supports asynchronous I/O and has sentinels for exactly that. However, not everything that runs on Emacs is taking advantage of that functionality. It is therefore up to you, to use the proper package (or not use the wrong one) / configure Emacs in such a way, so that this sort of issue is avoided. I can definitely say that if you're…

This is putting a lot of blame on the user, who likely has no clue how emacs i/o behaves. > if you're experiencing I/O hangs, it's either misconfiguration or bad code in a package that you loaded I mean...yes. That's the issue. The issue is that it's possible to have i/o hangs caused by misconfiguration or elisp packages. The solution isn't to tell the user to do something different, it's to make it so that elisp exe…

How would you make it impossible to make the UI unresponsive? Given that lots of packages directly want to do stuff with UI?
Post reply on HN