Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Leveraging Rust and the GPU to render user interfaces at 120 FPS

zed.dev

Post reply on HN
Print view View original

161–170 of 212 posts

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#161

Post by fxtentacle »

meindnoch wrote: post #117
fxtentacle wrote: post #79

Earlier quoted context omitted.

Just checked and even without any additional setup JUCE is using CoreGraphics which is using Metal under the hood. So yes, the platform-specific renderer is using GPU. Also, you can use OpenGL for GUI compositing, too, not only as a 3D context.

CoreGraphics doesn't use Metal. You're confusing it with CoreAnimation.

JUCE has a CoreGraphicsMetalLayerRenderer which I understood as using Metal to render CoreGraphics paths.

https://github.com/juce-framework/JUCE/blob/4e68af7fde8a0a64...

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#162

Post by fxtentacle »

Jasper_ wrote: post #160
fxtentacle wrote: post #158

Earlier quoted context omitted.

You linked to the software fallback renderer which can be used for cross-platform compatibility. But JUCE also has platform-specific rendering modules. CoreGraphicsContext::createPath will convert the CPU spline segments to CG spline segments which are then rasterized by CoreGraphics using Metal on the GPU. https://github.com/juce-framework/JUCE/blob/2b16c1b94c90d0db... And on Windows 7 and up it'll use the hardware-…

You mentioned using the OpenGL context, and this code is used by the OpenGL context. CoreGraphics does not use the GPU. Direct2D uses a approach which tesselates paths into triangles on the CPU. It is similar to the JUCE code in that the GPU is used for coverage, but it still does not natively render splines on the GPU.

Juce has a CoreGraphicsMetalLayerRenderer which I believe uses Metal to render CoreGrapghics primitives.

And yes, I agree that JUCE will tesselate the splines into straight lines on the CPU. So technically, they are not fully rendered on GPU. But in practice, the slow rasterization is done on GPU while tesselation only has a negligible performance overhead. For example, I heard that UE4->UE5 removed the GPU tesselation support because it's usually easier to just DMA a new mesh from the CPU on-demand. I would treat that as a strong argument that CPU tesselation is practically never the bottleneck.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#163

Post by rubymamis »

as-cii wrote: post #57
nottorp wrote: post #54

I don't understand. Why would you need to render a user interface constantly at 120 fps, instead of just updating it when something changes? Laptop batteries last too long these days? Electricity too cheap?

Hey nottorp. Antonio here, author of the post. Zed and GPUI use energy very judiciously and only perform updates when needed. The idea is that we can render the whole application within ~8ms, and that shows everywhere: from typing a letter to scrolling up and down in a buffer. However, if the editor is sitting there idle, we won't waste precious CPU cycles. Thanks for the feedback!

Will you allow developers access to your GUI framework? What about open-sourcing it?

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#164

Post by msvan »

Lots of negativity in here. I for one am excited about the prospect of an editor that is as responsive as I remember Sublime being back in the day, with the feature set I've come to expect from VS Code. An editor like this simply does not exist today, and betting on the Rust ecosystem is entirely the right choice for building something like this in 2023.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#165

Post by fxtentacle »

gizmo wrote: post #97
fxtentacle wrote: post #85

Earlier quoted context omitted.

You are correct, but so is the comment you replied to. The latency that matters for a text editor is how long it takes for a new keypress to show up. And that's usually a 32x32 pixel area or less.

That's not sufficient, though. You want the 0.1% case where you press undo a couple of times and big edit operations get reversed to be smooth. You have to hit your frame target when a lot is happening, the individual key press case is easy. It's just like a video game. A consistent 60fps is much better than an average frame rate of 120fps that drops to 15fps when the shooting starts and things get blown up. You spen…

In a video game, you'd just blur the shit out of everything else while you grant yourself a 2nd frame to re-render the rest of the content [1]. Or you do temporal remapping like the Oculus does if you rotate your head too quickly.

[1] https://developer.nvidia.com/blog/nvidia-vrss-2-dynamic-fove...

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#166

Post by Animats »

This seems like the wrong portion of the problem on which to spend time. This is a text editor. Performance problems with text editors tend to involve long files and multiple tabs. Refresh speed isn't the problem, although keyboard response speed can be.

I'd like to see "gedit", for Linux, fixed. It can stall on large files, and, in long edit sessions, will sometimes mess up the file name in the tab. Or "notepad++" for Linux.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#167

Post by fxtentacle »

ben-schaaf wrote: post #112
fxtentacle wrote: post #85

Earlier quoted context omitted.

You are correct, but so is the comment you replied to. The latency that matters for a text editor is how long it takes for a new keypress to show up. And that's usually a 32x32 pixel area or less.

The other latency that matters is how long it takes to draw each frame when scrolling through the code, which is usually most if not all of the framebuffer.

Agree. In that case, you need to cache the result of your CPU rendering in a GPU texture so that you can smoothly scroll it. But you can probably still get away with drawing all UI components on the CPU if you add a little bit of caching.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#168

Post by fxtentacle »

scotty79 wrote: post #98
dorkwood wrote: post #10

Earlier quoted context omitted.

> KiCad uses a technique using RGB for gradients so that the rendered glyphs can be anti-aliased without accidentally rounding sharp corners. This is known as a “multi-channel signed distance field”, or “msdf”. https://github.com/Chlumsky/msdfgen

Do you know any text editor that uses it for font rendering?

You would have to use that if you want smooth antialiased zooming in and out of text.

BTW, the KiCad schematics are pretty text-heavy. You typically write down all the parameters and IDs of all the electrical components.

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#169

Post by fxtentacle »

qbasic_forever wrote: post #131
fxtentacle wrote: post #2

"Inspired by the gaming world, we realized that the only way to achieve the performance we needed was to build our own UI framework" I'm surprised you did not look at "Dear ImGui", "Noesis", and "JUCE". All three of them are heavily used in gaming, are rather clean C++, use full GPU acceleration, and have visual editors available. Especially JUCE is used for A LOT of hard-realtime professional audio applications. "Wh…

Google has done a ton of work on GPU accelerated vector graphics too for Android and other platforms. Their skia library is pretty nice: https://skia.org/

Oops I forgot to mention that one. They also have a pretty interesting cross-platform GUI framework using it: https://flutter.dev/

Re: Leveraging Rust and the GPU to render user interfaces at 120 FPS

#170

Post by Jasper_ »

fxtentacle wrote: post #162
Jasper_ wrote: post #160

Earlier quoted context omitted.

You mentioned using the OpenGL context, and this code is used by the OpenGL context. CoreGraphics does not use the GPU. Direct2D uses a approach which tesselates paths into triangles on the CPU. It is similar to the JUCE code in that the GPU is used for coverage, but it still does not natively render splines on the GPU.

Juce has a CoreGraphicsMetalLayerRenderer which I believe uses Metal to render CoreGrapghics primitives. And yes, I agree that JUCE will tesselate the splines into straight lines on the CPU. So technically, they are not fully rendered on GPU. But in practice, the slow rasterization is done on GPU while tesselation only has a negligible performance overhead. For example, I heard that UE4->UE5 removed the GPU tesselati…

> Juce has a CoreGraphicsMetalLayerRenderer which I believe uses Metal to render CoreGrapghics primitives.

This class is part of a JUCE demo app, and you can read the source code to it if you want. [0] It uses CoreGraphics to render the graphics on the CPU, and then uploads it as a texture to the GPU so it can be used as a CAMetal layer. So, no, the graphics are still rendered on the CPU, with compositing being handled on the GPU.

I don't know why you're confidently saying something incorrect, it took me all of 2 minutes to find the source code and read it.

> For example, I heard that UE4->UE5 removed the GPU tesselation support

I know it's confusing, but GPU tessellation is a completely different thing. The word "tessellation" in graphics means "turn into triangles". In a 2D graphics context, we're turning splines and curves and 2D shapes into triangles. In a 3D graphics context, GPU tessellation refers to a control cage mesh which is adaptively subdivided. These two have nothing in common except that triangles come out the other side. I am not aware of anyone who has tried to use GPU tessellation to render 2D graphics.

GPU tessellation failed for a large number of reasons, but poor performance was one of them. So, you know, doing this sort of work efficiently on the GPU is still an open research problem. Just because it's not efficient to do it on the GPU does not mean the performance overhead is negligible. For rendering big complex vector graphics, tess overhead can easily outweigh rasterization overhead.

[0] https://github.com/juce-framework/JUCE/blob/4e68af7fde8a0a64...

Post reply on HN