Live data from Hacker News

ht: Headless Terminal

github.com

81–90 of 90 posts

Re: ht: Headless Terminal

#81
post #3

I think it is a really good idea to separate vt100 emulation "backend" from the UI. Then all terminal emulators could use a common implementation and just focus on displaying the text, instead of emulating quirks of 50-year old devices. Using JSON as RPC also seems like a good idea, especially when SSH-ing in, as the language server protocol has shown. That being said, I don't see this project doing anything (yet?) a…

FWIW WezTerm does seem to split much of its "generic terminal" functionality into separate published re-usable crates:

* https://lib.rs/gh/wez/wezterm/wezterm / https://crates.io/crates/termwiz

(Projects that build on this crate includes "ratatui" among others...)

In particular, I note at least these specific feature areas:

* "support functions for applications interested in either displaying data to a terminal or in building a terminal emulator": https://lib.rs/crates/termwiz

* "to help with parsing input received from a terminal": https://docs.rs/termwiz/latest/termwiz/input/index.html

* "Model a cell in the terminal display": https://docs.rs/termwiz/latest/termwiz/cell/index.html

* "parse escape sequences and attach semantic meaning": https://docs.rs/termwiz/latest/termwiz/escape/index.html

* "abstraction over a terminal device": https://docs.rs/termwiz/latest/termwiz/terminal/index.html

* https://docs.rs/termwiz/latest/termwiz/color/index.html

* "cross platform API for working with the psuedo terminal (pty) interfaces": https://lib.rs/crates/portable-pty

* "Low level escape sequence parser": https://lib.rs/crates/vtparse

So, seems like quite a lot there to help avoid re-inventing the wheel for terminal-related functionality (for andyk/ku1ik & others).

Re: ht: Headless Terminal

#82
post #65
post #60

Earlier quoted context omitted.

`expect` is absolutely geared towards scripting, as it's an extension of TCL. Though as far as getting a "current terminal view" `expect` has `term_expect`: https://core.tcl-lang.org/expect/file?name=example/term_expe...

Sorry, my wording wasn't very clear. I wasn't trying to imply that ht is more geared towards scripting than `expect` (in fact I'd say `expect` is more scripting-oriented being an extension of a scripting language) but rather that ht is more geared towards scripting the terminal as a UI than `expect`. Am I wrong about that? (I may very well be since I haven't used `expect` before)

Based on my understanding/recollection of `expect`, the concept is that you're scripting a command/process (or command sequence) via a "terminal connection" (or basic stdin/stdout), based on the (either complete or partial) "expected" dialogue response.

e.g.

1. make initial connect over ssh (e.g. spawn ssh cli process) 2. expect "login: " response 3. send "admin" 4. expect "password: " response 5. send "password" 6. expect "$ " 7. send "whoami\n" 8. etc etc

I guess that might in theory be possible to script a TUI with but I suspect it'd get pretty convoluted over an extended period of time.

(BTW I mentioned this in a comment elsewhere in this thread but check out https://crates.io/crates/termwiz to avoid re-inventing the wheel for a bunch of terminal-related functionality.)

Re: ht: Headless Terminal

#83

Thank you for providing a musl binary. https://github.com/andyk/ht/releases/expanded_assets/v0.1.1

Indeed, maybe musl will finally be the long-term compatible Linux native binary format to one day replace win64+wine.

glibc compatibility breakage is the bane of my existence.

The situation is made worse by GitHub pushing people to build against the default extremely recent glibc in "ubuntu-latest" for binary artifact builds rather than what should be done: building against the oldest glibc possible (you can still do that within a "ubuntu-latest" container).

glibc breaks "version compatibility" at times for the most IMO ridiculous reasons but also has support for running binaries linked against older glibc versions on newer glibc that just gets completely ignored.

Re: ht: Headless Terminal

#84

Thank you for providing a musl binary. https://github.com/andyk/ht/releases/expanded_assets/v0.1.1

Is it even feasible to provide gnu binaries? I only upload statically linked musl binaries as Linux releases for my cli stuff.

> Is it even feasible to provide gnu binaries?

It depends on one's definition of "feasible", I guess.

Building against the "most recent glibc pushed by GitHub" doesn't make it feasible but building against an older glibc (which you can still do on a container which is otherwise using a recent glibc) is at least somewhat feasible.

The primary benefit from building against musl is that it "coincidentally" means that even the people who don't care at all about supporting older systems end up doing so "accidentally" due to their desire to use musl (for, presumably, some other reason).

Re: ht: Headless Terminal

#85
post #57

Earlier quoted context omitted.

The immediate thought I had upon reading the description was "this would be great for Minecraft servers". Most of us running Minecraft servers on Linux have it wrapped in screen or tmux because the CLI is the only way to issue certain commands including stopping it properly. This could provide an alternative.

What I typically do is create a systemd service for game servers and attach a TTY. That way it starts with the rest of my web services, and linux already handles "i/o to processes" via files that other process can access(e.x. /run/minecraft/{stdin,stdout,stderr})

Would you mind expanding on that, or can you point me at some relevant documentation?

I've never seen a systemd service example for Minecraft which allowed for sending commands to the server CLI and seeing the result without involving screen/tmux/etc. The top result on Google just doesn't allow command input at all, running the service "headless", the one on the official MC wiki uses screen, and the only other options I've seen use RCON which is neither secure nor does it show the responses you'd get on the MC console.

If there's a way to run just the straight Minecraft JAR as a background service and still be able to interact with it in the occasional cases where I need to I'm very interested.

Re: ht: Headless Terminal

#86
post #3

I think it is a really good idea to separate vt100 emulation "backend" from the UI. Then all terminal emulators could use a common implementation and just focus on displaying the text, instead of emulating quirks of 50-year old devices. Using JSON as RPC also seems like a good idea, especially when SSH-ing in, as the language server protocol has shown. That being said, I don't see this project doing anything (yet?) a…

FWIW WezTerm does seem to split much of its "generic terminal" functionality into separate published re-usable crates: * https://lib.rs/gh/wez/wezterm/wezterm / https://crates.io/crates/termwiz (Projects that build on this crate includes "ratatui" among others...) In particular, I note at least these specific feature areas: * "support functions for applications interested in either displaying data to a terminal or in…

ht uses asciinema’s avt as the underlying terminal emulator, so there wasn’t much re-invented here really. It was mostly glueing avt, PTY and JSON RPC together.

Re: ht: Headless Terminal

#87
post #85

Earlier quoted context omitted.

What I typically do is create a systemd service for game servers and attach a TTY. That way it starts with the rest of my web services, and linux already handles "i/o to processes" via files that other process can access(e.x. /run/minecraft/{stdin,stdout,stderr})

Would you mind expanding on that, or can you point me at some relevant documentation? I've never seen a systemd service example for Minecraft which allowed for sending commands to the server CLI and seeing the result without involving screen/tmux/etc. The top result on Google just doesn't allow command input at all, running the service "headless", the one on the official MC wiki uses screen, and the only other option…

Basically, you setup the standard minecraft service and then create a "socket" in systemd to use as stdin for the process(relevant documentation in systemd.socket and systemd.exec).

For me this looks like

-- /etc/systemd/system/minecraft.service --

  [Unit]                                    
  Description="Minecraft server service"
  
  [Service]
  Environment=JAVA_HOME="/usr/lib/jvm/java-22-openjdk"
  WorkingDirectory=/home/steam/minecraft/1.20.6/
  ExecStart=/usr/lib/jvm/java-22-openjdk/bin/java -Xmx4096M -Xms1024M -jar /home/steam/minecraft/1.20.6/server.jar nogui
  User=steam
  Group=steam

  Sockets=minecraft.socket
  StandardInput=socket
  StandardOutput=journal
  StandardError=journal

  [Install]
  WantedBy=multi-user.target
-- /etc/systemd/system/minecraft.socket --

  [Socket]                                  
  ListenFIFO=%t/minecraft.stdin         
  Service=minecraft.service
-------------

What this will do is add a systemd dependency on minecraft.service to start minecraft.socket first(which creates the fifo `/run/minecraft.stdin`) then setup minecraft.service to listen to this socket for it's StandardInput(while leaving stdout and stderr pointing towards the journal).

The service can then be started and set to automatically start on boot(`systemctl daemon-reload && systemctl enable --now minecraft`). While running, data can be written to the socket file with `echo` and redirection(e.x. `echo "help" > /run/minecraft.stdin`), and the output will be visible in the journal(`journalctl -xef --unit minecraft.service`)

If you set stderr/out to go over the socket as well, then you can attach something like `screen` to it and use it like a typical TTY(or `telnet`).

This uses the file `/run/minecraft.stdin` as the socket, but the documentation for systemd.socket shows that this can also be a TCP port to listen for connections(and systemd.service shows using regular files, but then you have to manually set them up).

Re: ht: Headless Terminal

#88
post #85

Earlier quoted context omitted.

What I typically do is create a systemd service for game servers and attach a TTY. That way it starts with the rest of my web services, and linux already handles "i/o to processes" via files that other process can access(e.x. /run/minecraft/{stdin,stdout,stderr})

Would you mind expanding on that, or can you point me at some relevant documentation? I've never seen a systemd service example for Minecraft which allowed for sending commands to the server CLI and seeing the result without involving screen/tmux/etc. The top result on Google just doesn't allow command input at all, running the service "headless", the one on the official MC wiki uses screen, and the only other option…

Oh yea one more comment, this stdin redirection isn't really necessary in minecraft from the last decade.

The minecraft server has a built-in RCON server running on a separate port than can be enabled(https://wiki.vg/RCON), and once enabled can be interacted with an RCON client(like https://github.com/Tiiffi/mcrcon).

So instead of redirecting stdin to a systemd process, you can also just leave stdin disconnected and use the built-in RCON server to do commands every so often.

Re: ht: Headless Terminal

#89
post #48

Earlier quoted context omitted.

I wrote an integration testing framework which I wanted to integrate with a tool exactly like this so it could be used to, e.g. test a command line app like vim. Expect is what I tried to integrate with first. It falls over quite quickly with any kind of app that does anything mildly complicated with the terminal.

Interesting. When we decided to build ht we didn't compare it to expect (which I hadn't heard of or used) but I'm comparing the two now as they seem related. How exactly did `expect` fall over? From what I can tell, expect does not provide the functionality of a stateful terminal server/client under the hood for you so it isn't as easy to grab "text" screenshots of a Terminal User Interface, which is one of the main…

Screenshots was one thing that didnt work that I needed but I think lots of control characters used by command line apps also messed up pexpect.

I built my own probably not very good equivalent of your thing called icommandlib. I'm going to investigate ripping it out and replacing it with your tool.

Re: ht: Headless Terminal

#90
It's pretty interesting. I have actually stumbled over this yesterday, because I needed to figure out a way to automate a very specialised and ancient TUI application (think car rental software that is still used worldwide). Meanwhile I've wrote a crude parser for ANSI escape sequences and managed a virtualised screen representation; but I'm also having a fair bit of problems sending commands.

ht seems to be almost "there" for me, and would allow me to easily build a sequence of actions. However, it's kinda missing the color representation, which is also a problem for me: I need to read the color of a specific row/column to know that a specific menu item is selected correctly, and then proceed from this.

Let's see to what it can evolve later.

Post reply on HN