Live data from Hacker News

Desktop compositing latency is real

lofibucket.com

261–270 of 306 posts

Re: Desktop compositing latency is real

#261

Earlier quoted context omitted.

That's weird, I don't have that problem. What build of Win10 are you running? I wonder if it depends on the kind of keyboard you're using.

I think it somehow depends on whether the OS is installed on a spinning disk or an SSD. I know it's stupid but this is my theory based on observation. Would love for someone to verify it.

I'm sure a spinning disk would make it worse, but I do have an SSD (admittedly, a 2013 model and not the fastest). To answer the grandparent, build 15063 and I'm using the laptop keyboard.

I tried a few times and I can consistently make Windows miss at least the first keypress if I haven't opened the Start menu in a few minutes.

Re: Desktop compositing latency is real

#262

Earlier quoted context omitted.

This assumes you are synchronizing the updates with the monitor's refresh cycle, however if you aren't (and the major reason you see lag in compositors is because they do such synchronizations) then composition is indeed slower since it involves several more moving parts and the need to orchestrate the refresh of multiple windows (as opposed to the instant "i want to draw on the screen now" model that X11/Xorg withou…

Yeah, having to synchronize multiple windows is probably a pain. I guess that's a much smaller issue with a VR application (the OpenVR compositor supports overlays, but they're not used that often, and there's a clear priority to the "main" VR content) I guess a valid approach would be to double buffer all windows on the compositor side, and render the "stale" buffer of any window that fails to update within a specif…

Compositors already do that, you render into a compositor managed texture and the compositor simply uses whatever is there so applications can update at their own leisure.

Re: Desktop compositing latency is real

#263

Earlier quoted context omitted.

Wayland uses a composition model which is inherently slower than having direct access to the front buffer.

... and when you give people direct access to the front buffer, they write code that tears or generally scans out incomplete renders and users end up blaming the operating system. Compositing is a good thing, and in the vast majority of cases its latency isn't actually intrinsically higher than writing directly to the front buffer. Certainly its intrinsic latency is never higher than writing directly to the front buf…

But i want the tearing, or at least i prefer it to the latency that compositors impose! This is why compositors must not be forced and instead be a user option. I do not see why i have to suffer a subpar computing experience because of some clueless users.

I even force vsync off system-wide where possible (that is in Windows, in Linux i haven't seen such an option and even in Windows DWM ignores the setting).

Re: Desktop compositing latency is real

#264

Earlier quoted context omitted.

There isn't much of a difference between a GUI toolkit you'd find in a desktop application and the GUI framework you'd see in a game - the most likely difference will be that the game GUI will be redrawn every frame whereas the desktop GUI wont (and there are game GUI frameworks that cache their output to avoid redrawing the entire widget tree every frame). The difference when it comes to why games can be snappier is…

The funny thing is that there is no technical reason at all for compositing to have worse latency, even for games. Think about the actual operations that are involved. You certainly never want to render directly into the front-buffer (you'd end up scanning out partially rendered scenes). So you render to a back buffer. Which you then blit to the front buffer (assuming you're in windowed mode; in full-screen mode the…

There is a very good practical reason why the compositor is in no position to fix that even if it could theoretically be possible.

A major source for the compositor latency (or actually, the increased response time you get with a compositor) is that the "render to back buffer" (ie. compositor's texture, at the best case) and the "blit to the front buffer" (which happens by the compositor by drawing the window geometry) do not happen at the same time.

From a technical perspective it is perfectly possible for a compositor to create a tight integration between a program and the compositor itself: simply synchronize the program's updates with the compositor updates. Every time the program says "i'm done drawing" (either via an explicit notification to the compositor, via glXSwapBuffers or whatever), issue a screen update.

The problem however here is the compositor has to take into account multiple windows from multiple programs so you cannot have a single window dictating the compositor updates. Imagine for example two windows with animations running, one at 100fps and another at 130fps. Depending which window is active (assuming that the compositor syncs itself with the active window), it would affect the perception of other window's updates (since what the user will see will be at the rate of the foreground window's update rate). Moreover, beyond just the perception, it will also affect the other windows' animation loops themselves - if a background window finishes drawing itself and notifies the compositor while the compositor is in the middle of an update, the background window will have to wait until the update is finished - thus having the foreground window indirectly also affect the animation loops of the background windows. This can be avoided through triple buffering, but that introduces an extra frame of latency - at least for background windows.

So to avoid the above problems, what all compositors do is to decouple window update notifications from screen updates and instead perform the screen updates at some predefined interval - usually the monitor refresh rate and they do the updates synchronized to it. However that creates the increased response time you get with the compositor being on a few milliseconds behind the user's actions, with the most common example would be window manipulation like resizing and moving windows lagging behind the mouse cursor (which is drawn by the GPU directly, thus bypassing the compositor).

Hence the linked article recommending a 144Hz monitor to avoid this, although this is just a workaround that makes the problem less visible but doesn't really solve it.

Re: Desktop compositing latency is real

#265
post #130

I will gladly believe this is a real problem, but this page does not demonstrate that (at least not convincingly); the metric used is simply too poor. To quote: > I used my own hacky measurement program written in C++ that sends a keypress to the application to be benchmarked. Then it waits until the character appears on screen. Virtual keypresses were sent with WinAPIs SendInput and pixels copied off screen with Bit…

The time measured is artificial but not necessarily uninteresting. You can't measure the true end to end latency this way, but you can compare different user applications to see how much latency they add. It's not perfect but it is the best available way to measure latency of arbitrary applications without extra hardware. I built a browser latency benchmark based on a similar method: https://google.github.io/latency-…

I'm sure they correlate: but the thing with correlation is that you can have correlation even when there are whole classes of situations where the relationship doesn't hold.

On the same os +drivers I'd be willing to believe that this measure is almost certainly useful (even there, it's not 100%). But it's exactly the kind of thing that a different way of, say... compositing... might the implementation to work a little differently, such that you're comparing apples to oranges. If BitBlt simply gets access a little earlier or later in the same pipeline and hey presto: you've got a difference in software that is meaningless in reality.

Re: Desktop compositing latency is real

#266
post #76

Earlier quoted context omitted.

The little plastic bits always snap off. Ethernet is great for semipermanent wires but not very good for peripherals like portable hard drives that are constantly disconnected and reconnected.

What’s that to do with displays which aren’t unplugged frequently?

USB-C is used for all sorts of devices many of which are unplugged frequently.

Re: Desktop compositing latency is real

#267

Earlier quoted context omitted.

The issue with sending a tree of layers is that this prevents some important optimizations: avoiding overdraw with early Z becomes impossible, because your app doesn't know anything about the positions of the scrollable layers and so has to be conservative and paint all of their contents. So you trade a frame of latency for lots of overdraw, which is a tradeoff I'm not really comfortable making. (Note that today, alm…

> So you trade a frame of latency for lots of overdraw, which is a tradeoff I'm not really comfortable making. I would jump at the chance to make that trade. My 8 year old CPU is rarely taxed by normal usage. What's the point in having a faster computer if it feels slower?

Overdraw doesn't just use some CPU. If that would be all, it'd be an easy problem to solve. It causes extra data to be transferred over various buses, extra stops in a pipeline of operations that is already cramped trying to provide a frame every few ms, and probably other problems that I don't know about. This problem isn't as easy as 'throw some more CPU at it'.

Re: Desktop compositing latency is real

#268

Earlier quoted context omitted.

this reminds me of the argument that instead of going to USB C 3.1 as the grand unifying connector for all peripherals (including video), we should have instead migrated to Ethernet cables for everything.

Currently ethernet doesn't look cheap enough for video to me. Even a simple 1920x1080 60Hz monitor needs 3 Gb/s, a 2560*1440 144Hz monitor 13 Gb/s, a 4k 60Hz 12 Gb/s. DisplayPort 1.2 offers 17 Gb/s and DisplayPort 1.3 offers 26 Gb/s.

40 Gbps and 100 Gbps Ethernet standard was defined in 2010 [1]. Google "buy 100 Gbps Ethernet" and you will find it is real, although currently is geared primarily for datacenters.

[1] wikipedia.org/wiki/100_Gigabit_Ethernet

Re: Desktop compositing latency is real

#269
post #41
post #33

Earlier quoted context omitted.

"I can send an IP packet to Europe faster than I can send a pixel to the screen. How f’d up is that?" - John Carmack

Solution: send the pixel to the screen over IP! (joking, but with a tiny serious element)

Some people tried, most went out of business. Either chapter 11 or bought for assets and IP. OnLive was the biggest one.

Re: Desktop compositing latency is real

#270

Earlier quoted context omitted.

The funny thing is that there is no technical reason at all for compositing to have worse latency, even for games. Think about the actual operations that are involved. You certainly never want to render directly into the front-buffer (you'd end up scanning out partially rendered scenes). So you render to a back buffer. Which you then blit to the front buffer (assuming you're in windowed mode; in full-screen mode the…

There is a very good practical reason why the compositor is in no position to fix that even if it could theoretically be possible. A major source for the compositor latency (or actually, the increased response time you get with a compositor) is that the "render to back buffer" (ie. compositor's texture, at the best case) and the "blit to the front buffer" (which happens by the compositor by drawing the window geometr…

This "do not happen at the same time" is true in plain Xorg as well, though, since the final blit to the screen happens in the X server and not in the application.

Your example of 100fps vs. 130fps on the same screen is inherently unsolvable in a proper way with anything less than a 1300fps display. So you have a bunch of tradeoffs, and I'm sorry to say that if the tradeoff you prefer is tearing, you're in the losing minority by far.

That said, if you truly wanted to write a tearing Wayland compositor, you could easily do so, and in any case plain X is still going to work as well.

Post reply on HN