Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Design of a Vim-like text editor

lists.suckless.org

Post reply on HN
Print view View original

71–80 of 92 posts

Re: Design of a Vim-like text editor

#71

Post by ridiculous_fish »

rtpg wrote: post #70
ridiculous_fish wrote: post #69

This piqued my interest: > handle arbitrary files (this includes large ones, think >100M SQL-dumps) My old hex editor Hex Fiend was a serious attempt to handle arbitrary-sized files correctly. It's hard! In particular, operations which are usually instantaneous (e.g. Find) now may take a long time: they need progress reporting and cancellation, and ideally should not be modal. A text editor makes that even harder, be…

>For one thing, it means you cannot work with files larger than maybe 3 GB, or even 3 1 GB files, in a 32 bit process. Serious question : is it bad to assume a 64-bit capable running environment for developer's computers nowadays? I feel like it's better to run with something simple (and way less error prone) if the cost is some 32-bit issues like stated. Granted, vim is a special case, since we're running this throu…

There's definitely still new Linux systems that are 32 bit (Raspberry Pi, etc) though probably not many as a developer computer. Maybe Crouton on a Chromebook?

Anyways you have a point about the simplicity of mmap. Reading up on the suckless philosophy, it does seem to fit.

Re: Design of a Vim-like text editor

#72

Post by marvy »

ridiculous_fish wrote: post #69

This piqued my interest: > handle arbitrary files (this includes large ones, think >100M SQL-dumps) My old hex editor Hex Fiend was a serious attempt to handle arbitrary-sized files correctly. It's hard! In particular, operations which are usually instantaneous (e.g. Find) now may take a long time: they need progress reporting and cancellation, and ideally should not be modal. A text editor makes that even harder, be…

Um, how on earth do you deal with the 10GB delete? You can't store it in RAM, since there may not be enough, especially if you want 32 bit support. Store it on disk? You can't store it in the original file, since you have no flexibility over its contents after a save: a hex editor must write the bytes asked for, unlike a word processor which controls the file format. Ok, maybe we store it in some other file, call it "temp". That almost works, but what if the point was to save disk space? "Please restart the editor"? And if the power goes out one hour later, the user finds out at the next boot that clearing out a 10 gig file did not save any disk space?

I'm out of ideas. What does Hex Fiend do?

Re: Design of a Vim-like text editor

#73

Post by shrizza »

ridiculous_fish wrote: post #69

This piqued my interest: > handle arbitrary files (this includes large ones, think >100M SQL-dumps) My old hex editor Hex Fiend was a serious attempt to handle arbitrary-sized files correctly. It's hard! In particular, operations which are usually instantaneous (e.g. Find) now may take a long time: they need progress reporting and cancellation, and ideally should not be modal. A text editor makes that even harder, be…

You mentioned Hex Fiend; hed is also a decent hex editor for large files. I've edited 5G files in it.

Edit: Ah, just noticed you're the author. Hi.

Re: Design of a Vim-like text editor

#74

Post by ridiculous_fish »

marvy wrote: post #72
ridiculous_fish wrote: post #69

This piqued my interest: > handle arbitrary files (this includes large ones, think >100M SQL-dumps) My old hex editor Hex Fiend was a serious attempt to handle arbitrary-sized files correctly. It's hard! In particular, operations which are usually instantaneous (e.g. Find) now may take a long time: they need progress reporting and cancellation, and ideally should not be modal. A text editor makes that even harder, be…

Um, how on earth do you deal with the 10GB delete? You can't store it in RAM, since there may not be enough, especially if you want 32 bit support. Store it on disk? You can't store it in the original file, since you have no flexibility over its contents after a save: a hex editor must write the bytes asked for, unlike a word processor which controls the file format. Ok, maybe we store it in some other file, call it…

The short answer is that it preserves the undo stack, if it can do so without using more than 16 MB of additional memory. If it exceeds that threshold it drops the stack. (Hey, I didn't claim to have solved it well!)

The longer answer: when a file is going to be saved, HF identifies the ranges of the file that will be modified, and then identifies every other "byte array" (what vis calls a "piece chain") that references one of those file ranges. This includes data that may have been copied and pasted from the file, as well as the undo stack.

It then attempts to "break dependencies" on the about-to-be-overwritten ranges in the file, by copying the referenced bytes into memory first. But if that copying would exceed that 16MB threshold it gives up. Giving up in the case of the undo stack means silently dropping it, and in the case of pasting into another document, it prompts the user: http://i.imgur.com/6Vq7VyQ.png

The guiding UI principle is to avoid surprise allocations of large amounts of memory or disk space.

Re: Design of a Vim-like text editor

#75

Post by airblade »

_b wrote: post #44

Right now, using vim is better than using this new editor (vis), although vis is close to being substantially better than vim. I use the suckless window manager (dwm) and terminal (st), so gave vis a try for a couple days. Three vim plugins made me return: git-gutters, you-complete-me (ycm), and clang-format. It is hard to extend vim's functionality. Its plugin interface has issues. Because of that, ycm and git-gutte…

The reason ycm and vim-gitgutter don't work well together isn't the "plugin interface"; it's the limited nature of the sign column which can only display one sign per line. If both ycm and gitgutter want to display a sign on the same line, only one can succeed.

Edit: oops, ycm doesn't use signs...I was thinking of syntastic. As far as I know ycm and vim-gitgutter should work together.

Re: Design of a Vim-like text editor

#76

Post by edwintorok »

lobster_johnson wrote: post #68

The article mentions the Project Oberon book and the piece list algorithm, which goes back to Xerox PARC's Bravo editor. Coincidentally I came across this paper recently about different data structures for text editors, which covers the piece list algorithm but also several others: "Data Structures for Text Sequences" (1998), Charles Crowley ( https://www.cs.unm.edu/~crowley/papers/sds.pdf ).

There is also this old book about Emacs that describes data structures for text editing: http://www.finseth.com/craft/#c6

Re: Design of a Vim-like text editor

#78

Post by gchp »

kzrdude wrote: post #64
gchp wrote: post #58

Earlier quoted context omitted.

Another part of it which I didn't mention is that I've always wanted to build a text editor, they've always intrigued me. So this was me scratching the itch if you like. I also used it as an opportunity to learn rust, which I really like, incidentally. To answer your question, I actually hadn't found one that suited me, fully. There are many that I like, but non which did it for me. It sounds a bit weird, but it need…

I was curious if you could describe the specific frustrations, and which of these you can fix.

Right, here goes...

- ability to handle large files without freezing or without a noticible drop in performance

- make all aspects of the editor scriptable. Emacs does this well, but I really don't like elisp. It will also be possible to script it in a number of languages, not limited to a single one.

- fast, like really fast. My Emacs config takes ten seconds to load, using IDEs takes much more than that to open. I want to make use of modern processor capabilities and minimise load time. Also to parallelise as much of the editing functionality as possible, so the user is never blocked by a task or operation.

- cross platform, easily installable. Some editors do this well, others don't. With iota you'll be able yo just download a binary and run it. No installation process, no dependencies. Same process to install on all machines.

I guess those are the main ones. Am on mobile right now so its hard to give more detail at the moment. I'll try update the motivations part of the readme with information like this today.

Post reply on HN