Leveraging Rust and the GPU to render user interfaces at 120 FPS • Hacker News Board

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#171

Post by exo-cortex » Thu, Mar 09, 2023, 6:24 PM UTC

btryder wrote: post #151

Lots of people nit-picking the 120 FPS but I think Zed looks super promising. The native support for collaborative editing looks fantastic, and I'm excited to try it out. Curious if you guys have thought about VR / AR possibilities with GPUI?

That would be interesting in VR to have syntax highlighting that moves variable names towards the viewer, or puts context menus with possible methods of some object above.

Would probably just be stupid gimmick, but I want to see somebody do it :-D

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#172

Post by tracker1 » Thu, Mar 09, 2023, 6:24 PM UTC

seanalltogether wrote: post #38

yason wrote: post #17

The bottleneck of UI is not the rendering. A measly 60 fps is plenty fast for UI that feels immediate. We had this in the 90's with software rendering, you don't need a GPU for that today. What causes user interfaces to hick up is that it's too easy to do stuff in the main UI thread. First it doesn't matter but stuff does accumulate, and eventually the UI begins to freeze briefly for example, after you press a button…

Separating the logic thread from the render thread doesn't end up being the silver bullet to performance. If I press the letter 'a' on my keyboard and my IDE suddenly stalls on a bunch of code hinting logic, that new state is still not going to make it to the render thread for a couple frames. As long as the render thread is dependent on state coming in from the logic thread, the stalls still propogate. You could hav…

That's a big reason why VS Code pushed the extensions (for code completion logic, etc) to another process. Where VS proper was in-thread... man, some web projects in VS are just painful.

I do think that they will need to do something similar with a higher level language support for extensions. Could probably piggy-back the Deno work for using V8/JS/TS for extensions similar, maybe even compatible with VS Code extensions. It's a massive user space, but seeing the core feature set in another editor could be really nice.

For me the killer feature of VS Code is really the integrated terminal. Not having to switch windows, or leave the app to run a quick command is really big imo.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#173

Post by haberman » Thu, Mar 09, 2023, 6:27 PM UTC

This sounds really similar to the story we were hearing about Servo back in 2016: https://www.youtube.com/watch?v=erfnCaeLxSI

I was really excited when I saw that demo. Why didn't this turn into a final product that people could use?

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#174

Post by tiffanyh » Thu, Mar 09, 2023, 6:41 PM UTC

Nathan Sobo

It should be noted that the main person behind Zed is Nathan Sobo, who created Atom while he was at Github, which is the basis of Visual Studio Code today.

As such, I have high hopes Zed will be a much faster version of Visual Studio Code and am excited to see what him & his team make.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#175

Post by KptMarchewa » Thu, Mar 09, 2023, 6:44 PM UTC

DeathArrow wrote: post #22

According to Wiki, the technique was invented in 2005 by Casey Muratori: https://en.wikipedia.org/wiki/Immediate_mode_GUI

Some very dedicated fan must have added this. I can see 2001 Java doc about some immediate mode classes:

https://nick-lab.gs.washington.edu/java/jdk1.5b/guide/2d/spe...

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#176

Post by KptMarchewa » Thu, Mar 09, 2023, 6:46 PM UTC

avereveard wrote: post #67

I cannot stress how much I do not want my 300w gpu to be used to render text that change at most three times per second. And it's not just about electricity cost and heat stress, it will conflict with everything else that requires the gpu to do stuff, including watching 4k videos on the second monitor, which does have a legitimate case for requiring hardware acceleration since they move a lot of data 60 times per sec…

Processors could throttle themselves since the 90s. GPUs can too.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#177

Post by tayistay » Thu, Mar 09, 2023, 7:04 PM UTC

My rui library can render UIs at 120fps, uses similar SDF techniques (though uses a single shader for all rendering): https://github.com/audulus/rui

Is their GPUI library open source?

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#178

Post by tayistay » Thu, Mar 09, 2023, 7:12 PM UTC

29athrowaway wrote: post #141

You cannot see at 120 fps.

If you're doing a drawing app, then 120fps helps you keep the stroke close to the stylus. Even then it may lag a bit because of a few frames of GPU latency and you have to do predictive stroke points and then adjust your stroke later. See https://developer.apple.com/documentation/uikit/touches_pres...

Similarly, if you're dragging something around with your finger, 120fps keeps that thing closer to the finger. Just improves the UX.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#179

Post by meindnoch » Thu, Mar 09, 2023, 7:15 PM UTC

fxtentacle wrote: post #161

meindnoch wrote: post #117

Earlier quoted context omitted.

CoreGraphics doesn't use Metal. You're confusing it with CoreAnimation.

JUCE has a CoreGraphicsMetalLayerRenderer which I understood as using Metal to render CoreGraphics paths. https://github.com/juce-framework/JUCE/blob/4e68af7fde8a0a64...

Incorrect. Please read the code more carefully.

It uses CoreGraphics to render into a bitmap on the CPU, which is then uploaded into a Metal texture.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#180

Post by GreedCtrl » Thu, Mar 09, 2023, 7:30 PM UTC

mwcampbell wrote: post #101

Gigachad wrote: post #14

Earlier quoted context omitted.

I’ve heard it’s hard to even work this out as all of the screen reader tools are expensive, proprietary, and there are no standards. The typical way is to just make your program, and if it gets popular, the screen reader companies will find a way to make their product work.

This used to be the case on Windows, but hasn't been for at least 10 years, and has never been the case on Mac OS X (using the historical name for clarity) or Linux. On Windows, the open-source NVDA screen reader is widely used. Furthermore, the hacks that Windows screen reader developers historically used to "find a way to make their product work", particularly intercepting GDI API calls (either in user space or in…

As someone who's doing some accessibility programming but has no background in the field, I do have a random question to throw your way if you are game.

Our team is developing voice control for a website. This is mainly requested by sighted users who want to use the site hands-free. But we also have an accessibility mode for better screen reader support.

We think voice control might be appreciated by screen reader users too, but we aren't sure how well it would work with a screen reader.

Are there common pitfalls we should be wary of?

One thing we're worried about is that the voice from the screen reader might interfere with the detection of the user's voice.

I know general dictation and voice control software already exists, so my initial assumption that screen reader users would benefit might be wrong. If the existing tools are good enough, perhaps this whole question is moot.