Emacs Is Not Enough

> In some ways it is impressive that Emacs has managed to avoid being killed off in a massive rewrite attempting to chase other text editors.

No point chasing when you’re the leader.

It would have to happen in a fork, and a fork attempting it would run into all the problems Richard Stallman identifies when these things are proposed (and rejected). It’s not like they’re unaware that some decades old critical components need to be redone. But this is emacs, not a FAANG throwaway, so that kind of work has constraints and expectations and standards to meet. Chunking garbage at it because emotions isn’t an option there.

Yes, yes, yes and yes. I am glad to see this sentiment expressed here!

> trying to find a general structure and fit it for everything is a path to failure

Depends on whether you consider everything literal. That nothing can solve everything and your mother is obvious. But solutions which can handle majority, or even 99.99% of relevant cases of your domain, I think we already have that today for structures. XML for example was optimized for decades to handle structures. Not all parts are good for the majority of cases, but the general ideas and concepts are universal enough that flavors of them will always appear when tackling such problems.

So instead of build yet another new solution, maybe just look at the existing tools and how we can make them more useable for your usecases.

> PS I have been using vim for quite a few years, and I don't see it as some kind of complex editing (other than bindings and modality which takes getting used to). Maybe I am misunderstanding what you mean by this exactly, though.

Normal editing today is: you press a key, something happens, that's all. Sometimes you add a modifier, but that's mostly it. Vim on the other side, you have modes, you have parameters for key presses, you compose commands to create new commands on the fly. That all is several steps more complex than just a simple key press. Vi-Input is a whole language in itself, highly complex on a cryptic level, all just to macro-manage micro-tasks.

this article is hilarious because the vim csv plugin 'just works.'

I like the fact of make people's lives easier. It's a universal solution which may be used in many editors: can't argue against reuse.

But is it perfect? I don't think so. Whatever is responsible for structural analysis should also be responsible for editing and all the extension capabilities, if you are to have a truly powerful system.

For me, the real problem lies in the integration with the client environment. Tree-sitter is fundamentally separate from the client application. That places questions on the introspection capabilities.

I mean, suppose your comment has structure. Suppose it's a markdown document, that comment in your code. And suppose you want to edit it that way: as a markdown document (or anything more complex). So you may want a markdown parser for that. Suppose you can write that parser in tree-sitter. Suppose you can even do structural-embedding. But how are you going to control and customize it? How are you going to write complex interaction policies between different structures? Embedding is where such seperation starts running into limits.

Why does this happen? I believe it's simple: APIs aren't enough. What you want is for the user to have access to the building blocks of whatever is being manipulated. And, so, keeping those blocks away from the user is never going to be that.

What's more, tree-sitter is still not really structural. That means it can't handle ambiguity well. For instance, in a lisp expression, you can insert a stray parenthesis. Paredit or a tree-sitter solution is going to tell you that your structure is malformed. A truly powerful structural editor is going to simply assume that parenthesis doesn't disrupt the structure around it, and so, it will simple keep track of it, and maybe it will find a match within the same expression at some later point in time.

Yes: a structural lisp editor doesn't have to care about a stray parenthesis. It just keeps keeping everything around it intact. It's a local ambiguity.

And, at last, the very important idea of structural editing is not even editing: it's the ability to treat structures as objects. It's the ability to present objects using a textual interface. Suppose you have a table structure, multiple layers of it. You need direct access to all that. Editing it should be easy, but it's really secondary.

Tree sitter doesn't deal with this.

I still see tree-sitter as useful, because you still need to deal with all the plain-text files out there. Perhaps, it could be used to import plain-text into a structural-editor representation, and then it may be, again, used on small subexpressions to identify their nature, incrementally, at runtime (but not the whole document, like what a usual incremental parser would do). So, that's the value of it as a parsing tool.

So, yeah, it's a useful tool, but when it comes to some advanced points of power-use and extension, it's not really enough.

Thanks for the question, it's thought provoking in many ways.