Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Leveraging Rust and the GPU to render user interfaces at 120 FPS

zed.dev

Post reply on HN
Print view View original

101–110 of 212 posts

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#101

Post by mwcampbell »

Gigachad wrote: post #14
bruce343434 wrote: post #13

Earlier quoted context omitted.

If you want to do this, where can you start? What are some patterns for making code that's not too spaghetti when you have to handle tabbing, focus, layout, speech of element contents, the actual hierarchy of the elements etc? Are there standardized OS accessibility API hooks or something?

I’ve heard it’s hard to even work this out as all of the screen reader tools are expensive, proprietary, and there are no standards. The typical way is to just make your program, and if it gets popular, the screen reader companies will find a way to make their product work.

This used to be the case on Windows, but hasn't been for at least 10 years, and has never been the case on Mac OS X (using the historical name for clarity) or Linux. On Windows, the open-source NVDA screen reader is widely used. Furthermore, the hacks that Windows screen reader developers historically used to "find a way to make their product work", particularly intercepting GDI API calls (either in user space or in a display driver) to build an off-screen model, are not applicable to modern UI stacks. And the other major screen reader hack, using application-specific COM object models, was mostly only applicable to Microsoft Office and Internet Explorer. So you basically have to implement platform accessibility APIs to make your UI accessible. (If you use native controls, that's more or less done for you.)

Edit: BTW, I've been in the assistive technology industry a while, particularly on Windows. Feel free to ask me anything.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#102

Post by jcelerier »

ben-schaaf wrote: post #47
yason wrote: post #17

The bottleneck of UI is not the rendering. A measly 60 fps is plenty fast for UI that feels immediate. We had this in the 90's with software rendering, you don't need a GPU for that today. What causes user interfaces to hick up is that it's too easy to do stuff in the main UI thread. First it doesn't matter but stuff does accumulate, and eventually the UI begins to freeze briefly for example, after you press a button…

Unfortunately this isn't true anymore when you get to very high resolutions like 8k. Just clearing the framebuffer of an 8k display at 60hz requires ~6GB/s; about 1/10th of the theoretical memory bandwidth of modern desktop processors. Add in compositing for multiple windows, text rendering and none of that likely being multi-threaded it's pretty clear a CPU has no chance of keeping up.

But when you do software rendering you don't need to redraw the entire frame buffer, just the parts that changed, which can be very small even at 8k - e.g. the box of a checkbox being switched

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#103

Post by doodlesdev »

fassssst wrote: post #88

Erm, native WinUI apps are GPU accelerated and render at vsync.

Also GTK4 is GPU accelerated whenever possible, with really well mantained Rust bindings for it I think the only thing missing would be macOS which I'm not sure what solutions are there.

Another option in this front could be Flutter if write-once run-everywhere is a need for the project. Another advantage is that it's not only GPU accelerated but it's also retained mode.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#104

Post by pornel »

pjmlp wrote: post #68
nicoburns wrote: post #34

Earlier quoted context omitted.

It’s hard to make a cross-platform UI that way.

That is what wrappers and platform plugins are for, no need to build a full blown API from scratch.

Such solutions are often in tension between using only the lowest common denominator, and the code having different implementation for each platform anyway.

For example, their editor has tabs for editor buffers. Cocoa has a static tabbed widget, which has wrong look and odd UX for this. Cocoa also has a tabbed window type, which isn't a widget you can control. I imagine it'd be hard to abstract that away to work consistently with how Windows does tabbed views. I also haven't seen Windows' tabs being draggable, so that would probably need special DIY solution for Windows which Cocoa tabbed windows don't need.

Anyway, I think native UI toolkits are dying. For most people the Web is their most familiar "toolkit" now, and native platforms instead of fighting that back with clear consistent design, went for flat design and multiple half-assed redesigns that messed up all remaining expectations of how "native" looks and feels.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#105

Post by flohofwoe »

jspdown wrote: post #62
marginalia_nu wrote: post #41

Earlier quoted context omitted.

ImGui is amazing to work with though. Like holy fuck is it pleasant compared to basically every other UI-development paradigm ever in the history of user interfaces.

Yes but that's because it's an immediate mode renderer. It feels intuitive and that's why it's the de facto standard for building debug tooling. But this intuitiveness comes at a price! No matter how hard you try, this will always be slower and computationally heavy compared to retained mode. Great for some stuff, terrible choice for some other.

The term "immediate mode UI" is about the API philosophy, not about how it works under the hood, and especially not how rendering is implemented (e.g. Dear ImGui records a command list for the whole frame which is then passed to the API user for rendering, this is pretty much the opposite of an 'immediate mode renderer').

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#106

Post by bobajeff »

ohgodplsno wrote: post #75
fxtentacle wrote: post #2

"Inspired by the gaming world, we realized that the only way to achieve the performance we needed was to build our own UI framework" I'm surprised you did not look at "Dear ImGui", "Noesis", and "JUCE". All three of them are heavily used in gaming, are rather clean C++, use full GPU acceleration, and have visual editors available. Especially JUCE is used for A LOT of hard-realtime professional audio applications. "Wh…

As for text rendering, Slug [0] has existed for much more than ten years, and is pretty much the gold standard for GPU text rendering. [0] https://sluglibrary.com/

Thanks, I'm reading the paper* linked to on that site explaining the technique used in the library. It's encouraging me more to pursue implementing a 2D render on the GPU. I'm also inspired by a recent talk about gkurve**.

* https://jcgt.org/published/0006/02/02/

** https://m.youtube.com/watch?v=QTybQ-5MlrE

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#107

Post by doodlesdev »

mariusmg wrote: post #37
DeathArrow wrote: post #20

What's wrong with using platform APIs? I think that by 2023 most UI toolkits provided by the OS are hardware accelerated.

If their plan is to make their app crossplatform (Windows,OSX,Linux) and be very versatile with the UI customization , then maybe writing a small/focused specifically for your needs UI crossplatform toolkit is not such a bad idea (after all this is what Sublime Text is doing as well) But the HW accelerated brag is pointeless. Even if they manage to squeeze some extra performance over native toolkits, that is not nece…

   > Drawing the UI is never the bottlneck in a text editor...
Unless you're using a webview, which is... unfortunately the case for some of the popular code editors available. Sad times we live in.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#108

Post by flohofwoe »

nottorp wrote: post #54

I don't understand. Why would you need to render a user interface constantly at 120 fps, instead of just updating it when something changes? Laptop batteries last too long these days? Electricity too cheap?

It's not about rendering static screens at 120Hz, but rendering anything that's animated at a smooth 120Hz.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#109

Post by ohgodplsno »

bobajeff wrote: post #106
ohgodplsno wrote: post #75

Earlier quoted context omitted.

As for text rendering, Slug [0] has existed for much more than ten years, and is pretty much the gold standard for GPU text rendering. [0] https://sluglibrary.com/

Thanks, I'm reading the paper* linked to on that site explaining the technique used in the library. It's encouraging me more to pursue implementing a 2D render on the GPU. I'm also inspired by a recent talk about gkurve**. * https://jcgt.org/published/0006/02/02/ ** https://m.youtube.com/watch?v=QTybQ-5MlrE

Do note that as far as I know, this technique is patent-encumbered, at least in the US.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#110

Post by doodlesdev »

soulbadguy wrote: post #43

I am curious how this would compare to a ui written in flutter. It seems that fluter is also hardware accelerated and x-platform

In terms of performance it would possibly be even better in Flutter, BUT (and this is a big butt) text editing on the desktop in Flutter currently can only be described as broken. I last looked at this a few months ago and I think it's fixable, but as much as I like Flutter I don't think it's a good option for a text editor _just yet_.
Post reply on HN