Live data from Hacker News

Desktop compositing latency is real

lofibucket.com

241–250 of 306 posts

Re: Desktop compositing latency is real

#241

Earlier quoted context omitted.

How are you testing the mouse cursor following the window borders on resize? I've read that windows turns off the hardware sprite mouse cursor when resizing windows so that it can software render it to always line up properly.

Visual inspection for now. I've got an Arduino and a high-speed camera, so my plan for the next step is to send mouse and keyboard events from the Arduino, blinking an LED at the same time, then capture both the LED and the monitor in the video. Then a bit of image analysis. This is the only way to be quantitative and capture all the sources of latency.

Would you be testing this with a high refresh rate monitor (120hz+) ? Would that matter?

Re: Desktop compositing latency is real

#242
"The big problem with latency is that it accumulates. Once some component introduces delay somewhere in the input chain you aren’t going to get it back. That’s why it’s really important to eliminate latency where you can." - a lesson that applies to many things besides the narrow case of Windows 10.

Re: Desktop compositing latency is real

#243

I've been experimenting with this too, in the context of the Windows front-end for xi editor. It's absolutely true that the compositor adds a frame of latency, but I have a very different take than "turn it off." First, it's possible to design an app around the compositor. Instead of sending a frame to the system, send a tree of layers. When updating the content or scrolling, just send a small delta to that tree. Fur…

How are you testing the mouse cursor following the window borders on resize? I've read that windows turns off the hardware sprite mouse cursor when resizing windows so that it can software render it to always line up properly.

I also believe that is true.

My reason being, I had adjusted the screen brightness using software - but the mouse cursor is still brighter than white anywhere else. When I dragged a window, the cursor turns dim.

Re: Desktop compositing latency is real

#244

Earlier quoted context omitted.

Visual inspection for now. I've got an Arduino and a high-speed camera, so my plan for the next step is to send mouse and keyboard events from the Arduino, blinking an LED at the same time, then capture both the LED and the monitor in the video. Then a bit of image analysis. This is the only way to be quantitative and capture all the sources of latency.

Would you be testing this with a high refresh rate monitor (120hz+) ? Would that matter?

Yes and yes. My son has a 144Hz gaming monitor, and my main monitor for coding is a Dell 4k. We'll test both.

Re: Desktop compositing latency is real

#245
post #56

Earlier quoted context omitted.

Numerous games, including those having complex graphics and behavior, can render 120+ frames per second and realtime interactions (physics, optics, reactions) on pretty average hardware. I don’t think that game scripters who make final things like scenery or ui face complexity much harder than those in gtk/qt/wpf/htmljs widget programming. Details would be interesting though, since I’m no game developer. If true, it…

There isn't much of a difference between a GUI toolkit you'd find in a desktop application and the GUI framework you'd see in a game - the most likely difference will be that the game GUI will be redrawn every frame whereas the desktop GUI wont (and there are game GUI frameworks that cache their output to avoid redrawing the entire widget tree every frame). The difference when it comes to why games can be snappier is…

The funny thing is that there is no technical reason at all for compositing to have worse latency, even for games.

Think about the actual operations that are involved. You certainly never want to render directly into the front-buffer (you'd end up scanning out partially rendered scenes). So you render to a back buffer. Which you then blit to the front buffer (assuming you're in windowed mode; in full-screen mode the compositor goes out of the way anyway and lets you just flip).

The only difference between the various modes of operation is who does that final blit. In plain X, it's the X server. In Wayland, it's the Wayland compositor. In X with a compositor, it's the compositor.

Now granted, some compositors might be silly and re-composite the whole screen each frame, but clearly that can be avoided in most cases.

Depending on the setup, there can also be some issues with the scheduling of the game's jobs vs. the compositor's jobs on the GPU. Valve are working on this currently for VR, since the problem is much more noticeable there -- clearly it can be fixed one way or another (on Radeon GPUs you could do a blit via asynchronous compute if need be, for example), but note that compositing actually doesn't change this issue (since the X server's jobs also need to compete with the game's jobs for scheduling time).

So if compositing has worse latency, it's because nobody has cared enough to polish the critical paths. Conversely, compositing clearly does have advantages in overall image quality. So why not fix the (entirely fixable) technical problems with compositing?

Re: Desktop compositing latency is real

#246

Earlier quoted context omitted.

> "or GNU/Linux desktop" I believe Wayland has made some latency improvements.

Wayland uses a composition model which is inherently slower than having direct access to the front buffer.

... and when you give people direct access to the front buffer, they write code that tears or generally scans out incomplete renders and users end up blaming the operating system.

Compositing is a good thing, and in the vast majority of cases its latency isn't actually intrinsically higher than writing directly to the front buffer. Certainly its intrinsic latency is never higher than writing directly to the front buffer if you build a system without visual artifacts. (Because at the end of the day, all compositing does is shift around who does the job of putting things on the front buffer; the jobs themselves stay the same for all practical purposes.)

Re: Desktop compositing latency is real

#247

Earlier quoted context omitted.

There isn't much of a difference between a GUI toolkit you'd find in a desktop application and the GUI framework you'd see in a game - the most likely difference will be that the game GUI will be redrawn every frame whereas the desktop GUI wont (and there are game GUI frameworks that cache their output to avoid redrawing the entire widget tree every frame). The difference when it comes to why games can be snappier is…

I'll take a frame of latency and Wayland's not-completely-insane model over X any day.

yet strangely, wayland removes the frames-of-latency known as 'remote windowing' in it's quest for 'improvement'..

Re: Desktop compositing latency is real

#248

Earlier quoted context omitted.

ethernet over multimode fiber, I want to be able to have accessories hundreds of meters away from my computer

Just make sure those cables are never bent beyond the minimum bend radius.

Just use bend-insensitive fiber. It's not like copper cables like being kinked, either.

Re: Desktop compositing latency is real

#249
WOW! I usually run about 7 emulators of android in one monitor, and my web browser and other stuff in the other monitor. things get really laggy sometimes, turning off this feature made HUGE difference. my experience has improved dramatically

Re: Desktop compositing latency is real

#250
post #221

Earlier quoted context omitted.

Ghetto latency test: finger-scroll alternatingly up and down very quickly, see at which frequency your finger and scroll position are 180° out of phase, i.e. you finger is up while the contents are down or vice versa. Smartphones seem to be fine according to that test. Android is very good and iOS is even better.

Hm I get about 4 up and downs per second before the scroll position is 180deg out of phase in Safari on iPhone 7+. That translates to about 125 (1000/8) ms latency?

Yes, that is how it works :) That value seems surprisingly bad. My limited experience with iDevices (don't own one) has been that the position of the stationary finger on the page vs. finger on the page while scrolling (another way to measure - unless it is specifically fudged with some kind of prediction to make scrolling feel less detached) is very small. But I can't argue with data. FWIW, I like to test Android in the scroll view of the OS settings app or the address book. Those are well implemented and presumably don't add unnecessary lag.
Post reply on HN