Live data from Hacker News

Ratty – A terminal emulator with inline 3D graphics

ratty-term.org

201–210 of 262 posts

Re: Ratty – A terminal emulator with inline 3D graphics

#201
post #62
post #26

Terminal is slowly becoming a full featured web browser.

Isn't there a Terminal that renders everything with React? Super slow, but I guess thats what web devs want.

Do you one better.

Rendered as instanced quads in 3d space. Tens of millions at >60fps, and an entire class of "grid based text rendering" evaporates.

https://github.com/tikimcfee/glyph3d-js

Re: Ratty – A terminal emulator with inline 3D graphics

#203
post #25

IMO, next crazy step is for terminal to just have wayland or X11 protocol ? (/s or not?)

I have been thinking about this for a while. It's not as crazy as it may sound, especially in light of the other comments making a parallel between terminals and notebooks.

A few thoughts:

1. Linux VTs kind of have this feature already: there is the normal buffer, the alternate buffer (that something like htop would draw on), and an IOctl can change them to/from graphics mode.

2. It makes sense for interactivity. Kitty's graphics protocol is quite useful for static shapes, can be abused for animations, but doesn't really cut it for interactivity (say, pan a graph around). Wayland is designed for this.

3. Wayland would be a good fit: isolate each command from another, let them request buffers, but keep control of where to display them, do not update them when off screen, etc.

4. One downside is that terminals excel for one-shot tasks. What's the purpose of the display when you are done with it? Should you kill the process driving it? Due to this, it may make more sense to delegate more features to the terminal emulator (displaying the 3D model, etc). Or maybe just allow the app to temporarily take over the window.

5. Once you have it up and running, have it talk directly to the direct rendering manager. Your "kmscon" is now your compositor / desktop environment. That's a fun thought! Add some basic terminal features like tabs and tiling, and you've inverted the usual setup.

6. One downside is accessibility. I really like that I can copy-paste any part of the interface for reference, "screenshots", etc. It's good for screen readers, too. You lose these advantages by going to Wayland.

7. Another current terminal limitation is fonts. Power line, yazi & other make use of custom fonts for drawing part of the interface, logos, etc. AFAIK there is no good way to query their availability (which is also an issue for color emoji). Custom fonts or a new protocol could be useful, but client apps could draw it themselves if given a surface (that can already do that with the kitty graphics protocol, mind you)

Obviously I am not seriously considering to make such a terminal emulator, but it would be an interesting experiment (heck, maybe something I should try this "vibe coding" with, since I wouldn't want to spend too much time on it).

Re: Ratty – A terminal emulator with inline 3D graphics

#204

I like this. No reason the terminal should only support text. Data science notebooks show one way the terminal can evolve. Lots of interesting stuff happening in this space, with Kitty probably being the most aggressive innovator here [1]. I'm not sure there is an overall vision, though. [1]: https://sw.kovidgoyal.net/kitty/protocol-extensions/

I wonder if something like this could work for thumbnails in the terminal; I prefer to browse my filesystem from a terminal rather than the point and click file manager typically, and it would be really useful if I could have a grid-style `ls` with terminal based renders of the 3d models (thinking STL/STEP, 3D printing) in that directory. Bonus points if I could preview/rotate the model to inspect it.

yazi[0] seems to be able to display images iirc. Does it work for your use-case, hope this helps ya!

[0]: https://github.com/sxyazi/yazi

Re: Ratty – A terminal emulator with inline 3D graphics

#205

A couple of comments here mention using this in VR. Fwiw, years back I played a bit with shallow-3D UIs for software dev. Shallow like within a few cm of a laptop display, to minimize VAC eye strain for all-day use. Think more being able to layer and draw in color, but in 3D, rather than waving arms in a room. The 3D can be wiggle 3D, or perspective from webcam head/eye tracking, or stereo from shutter glasses, or XR…

I have a working fully 3D glyph based text rendering system I can't seem to get people to look at. It's this. Every character is a 3d placed quad, instanced rendered, so you get tens of millions and then some. They are individually addressable and mutable like any polygon. I use it to render entire GitHub repos in one go. I have two versions, native Apple and web. Web has the basics of an ide setup. Would love insigh…

it opens source files in an unreadable small size, presumably to fir the whole file into the window. i can zoom in, but i can't properly scroll around or select text. and i don't see the benefit of using 3d here. it doesn't seem useful.

Re: Ratty – A terminal emulator with inline 3D graphics

#206
post #91

I like this. No reason the terminal should only support text. Data science notebooks show one way the terminal can evolve. Lots of interesting stuff happening in this space, with Kitty probably being the most aggressive innovator here [1]. I'm not sure there is an overall vision, though. [1]: https://sw.kovidgoyal.net/kitty/protocol-extensions/

No evolution necessary! With my project, euporie [1], you can have use your data science notebooks with graphical image outputs, HTML, LaTeX, etc, all in the terminal. [1] https://github.com/joouha/euporie

This is such an amazing project. I find it so awesome that I can bump on such projects (and their creators, Hi!) on hackernews.

I wish to ask a question if I may (and as such pardon my ignorance on jupyter kernel, I don't know much about it and I hope you can tell me more about it :-D)

but my question is, is there a way to swap the jupyter kernel within euphorie to something else more minimalist?

And when you run a project with ssh, there are ways to give access to other users with user:password if I may ask?

I didn't know that there were ways to run jupyter kernels in terminal, I don't know when I might need it but I am prepared with this information now, this feels so nice to me, thanks for making it!!

This is like a checklist of a thing I didn't know that I needed/existed but the second I know that it has existed, it feels like my mind has checked it off and just a satisfaction from knowing projects like these existing.

(I think in some sense this is a bit of same reaction to me on Ratty too), Its just so good seeing projects in these spaces :-D

Edit: just remembered the one time I think I was using some websites which gave me jupyter and then I tried to use browsh to run jupyter to run jupyter in terminal so that it can be controlled by terminal but it had some issues and I wasn't able to run it.

I also wish to ask if there is a way to sign in to jupyter instance like that itself perhaps? (IIRC it was a jupyterhub instance)

Re: Ratty – A terminal emulator with inline 3D graphics

#207

A couple of comments here mention using this in VR. Fwiw, years back I played a bit with shallow-3D UIs for software dev. Shallow like within a few cm of a laptop display, to minimize VAC eye strain for all-day use. Think more being able to layer and draw in color, but in 3D, rather than waving arms in a room. The 3D can be wiggle 3D, or perspective from webcam head/eye tracking, or stereo from shutter glasses, or XR…

I have a working fully 3D glyph based text rendering system I can't seem to get people to look at. It's this. Every character is a 3d placed quad, instanced rendered, so you get tens of millions and then some. They are individually addressable and mutable like any polygon. I use it to render entire GitHub repos in one go. I have two versions, native Apple and web. Web has the basics of an ide setup. Would love insigh…

I opened and it said to use a repo tab. There are no tabs so I pushed a button. It had a list with Linux so I clicked that. It redirected to a 404 page on github and I gave up.

I don’t know what this is supposed to do, let alone how to use it. But I looked at it for you!

Re: Ratty – A terminal emulator with inline 3D graphics

#208

A couple of comments here mention using this in VR. Fwiw, years back I played a bit with shallow-3D UIs for software dev. Shallow like within a few cm of a laptop display, to minimize VAC eye strain for all-day use. Think more being able to layer and draw in color, but in 3D, rather than waving arms in a room. The 3D can be wiggle 3D, or perspective from webcam head/eye tracking, or stereo from shutter glasses, or XR…

I have a working fully 3D glyph based text rendering system I can't seem to get people to look at. It's this. Every character is a 3d placed quad, instanced rendered, so you get tens of millions and then some. They are individually addressable and mutable like any polygon. I use it to render entire GitHub repos in one go. I have two versions, native Apple and web. Web has the basics of an ide setup. Would love insigh…

looked at it. I don't think it works, as the other person said.

Re: Ratty – A terminal emulator with inline 3D graphics

#210
post #96

It's very interesting to learn about the newly proposed glyph protocol [1] in the linked blog post. I was bemoaning the lack of exactly this here about 6 months ago [2]! [1] https://rapha.land/introducing-glyph-protocol-for-terminals/ [2] https://news.ycombinator.com/item?id=45805072

The idea that every application should ship their own glyphs because some proprietary systems do not have normal fonts, is not great. Fix the terminal instead.

Also, if you want advanced GUI with icons, maybe you should just write a GUI app.

As for shipping custom icons, this is not very bright idea as well. If you switch between several applications on one terminal, then one application can redefine glyphs from another application. Also, when the application terminates, nobody cleans up its glyphs. Also, this increases attack surface because font standards are pretty complicated and one will be able to attack the system by just providing a glyph. We already have programs that can break the terminal, which should never happen.

Also, as for icons, I find emoji characters too distracting (and too large). They stand out too much and take away user's attention, breaking any visual hierarchy. The icons in terminal should be monochrome, and with thin lines, so that they do not distract you from the text and its structure.

Post reply on HN