Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Desktop compositing latency is real

lofibucket.com

Post reply on HN
Print view View original

121–130 of 306 posts

Re: Desktop compositing latency is real

#121

Post by pjc50 »

raphlinus wrote: post #98

I've been experimenting with this too, in the context of the Windows front-end for xi editor. It's absolutely true that the compositor adds a frame of latency, but I have a very different take than "turn it off." First, it's possible to design an app around the compositor. Instead of sending a frame to the system, send a tree of layers. When updating the content or scrolling, just send a small delta to that tree. Fur…

> Instead of sending a frame to the system, send a tree of layers.

Which API does one use to do this?

I'm slightly surprised that hardware overlays aren't already a feature, especially given that they're handled by the graphics card. I know there's a special API for video overlay, especially DRM video (where part of the requirement is that the system doesn't allow it near the compositor where it could be screenshotted). Can you do video "genlock" on Windows? (edit: https://msdn.microsoft.com/en-us/library/windows/desktop/dd7... )

I'm also wondering how VR/AR stuff handles this on Windows.

Re: Desktop compositing latency is real

#122

Post by jd3 »

further reading:

https://pavelfatin.com/typing-with-pleasure/

https://danluu.com/keyboard-latency/

In Windows 7, classic mode disabled the DWM compositor and V-Sync. It's incredibly dumb that Microsoft would arbitrarily remove that feature in Windows 10 to push their ugly as sin post-metro UI.

Re: Desktop compositing latency is real

#123

Post by pjc50 »

wruza wrote: post #56
fredley wrote: post #37

Earlier quoted context omitted.

Layers of abstraction take you further away from the metal. The more layers of abstraction your keypress must traverse before rendering is complete and the photons have reached your retina, the longer it will be until that happens. Layers of abstraction make complex tasks more reachable by a larger number of programmers by reducing the amount of specialist knowledge about those lower layers required to do the job. Th…

Numerous games, including those having complex graphics and behavior, can render 120+ frames per second and realtime interactions (physics, optics, reactions) on pretty average hardware. I don’t think that game scripters who make final things like scenery or ui face complexity much harder than those in gtk/qt/wpf/htmljs widget programming. Details would be interesting though, since I’m no game developer. If true, it…

> can render 120+ frames per second and realtime interactions (physics, optics, reactions) on pretty average hardware

Yes, but when you consider that the underlying "average" hardware of 2017 has a million times as many transistors and runs a thousand times as fast as the Amiga that seems less impressive.

Re: Desktop compositing latency is real

#124

Post by jsd1982 »

> Virtual keypresses were sent with WinAPIs SendInput and pixels copied off screen with BitBlt.

This methodology alone could account for the differences in timing between Win7 and Win10. For all we know, Win10 could just be slower at getting the pixels back to the program from BitBlt, or SendInput could be slower triggering events, or a multitude of other issues.

The best way to truly detect key-to-photon latency is with an external video recorder that has both the screen and keyboard in frame. Grant a few ms of noise for key travel distance.

Re: Desktop compositing latency is real

#125

Post by raphlinus »

pjc50 wrote: post #121
raphlinus wrote: post #98

I've been experimenting with this too, in the context of the Windows front-end for xi editor. It's absolutely true that the compositor adds a frame of latency, but I have a very different take than "turn it off." First, it's possible to design an app around the compositor. Instead of sending a frame to the system, send a tree of layers. When updating the content or scrolling, just send a small delta to that tree. Fur…

> Instead of sending a frame to the system, send a tree of layers. Which API does one use to do this? I'm slightly surprised that hardware overlays aren't already a feature, especially given that they're handled by the graphics card. I know there's a special API for video overlay, especially DRM video (where part of the requirement is that the system doesn't allow it near the compositor where it could be screenshotte…

DirectComposition. It's been there since Windows 8, and is used by Chrome among other apps (see https://bugs.chromium.org/p/chromium/issues/detail?id=524838).

It's possible some hardware already does overlays, given that the talk I linked above was 2 years ago. I haven't researched this carefully.

There's a bunch of stuff in the interface to support video and also integrated 3D content ("create swapchain for composition"), but I don't know how well it works. In my experiments and reading through the Chromium bug tracker, Microsoft's implementation of all this stuff is far from perfect, and it's hard, for example, to completely avoid artifacts on resizing.

Re: Desktop compositing latency is real

#126

Post by falcolas »

madez wrote: post #100
vetinari wrote: post #86

Earlier quoted context omitted.

The linked article explains it (TLDR: key travel time, scanning keyboard matrix, debouncing).

A keyboard doesn't need to implement a scanning matrix. It could hook up all keys individually to a an own IO-Port.

Requires a bigger chip (100+ IO pins) and more complex wiring diagrams than most inexpensive keyboard makers are willing to invest.

Re: Desktop compositing latency is real

#127

Post by [deleted] »

542458 wrote: post #74
dilap wrote: post #68

Earlier quoted context omitted.

in all seriousness, why not?

The obvious ones: Ethernet cables are big (imagine one on a phone), and aren't designed to be plugged and unplugged the massive number of times that USB cables are (spring contacts wear out, little clips break).

[deleted]

Re: Desktop compositing latency is real

#128

Post by badsectoracula »

TheCoreh wrote: post #102
badsectoracula wrote: post #79

Earlier quoted context omitted.

Wayland uses a composition model which is inherently slower than having direct access to the front buffer.

Not necessarily: video output is ultimately limited by the display. If the display runs at, say, 60Hz and both drawing on an off-screen buffer and compositing together take up less than ~16ms, the result should be exactly the same as drawing directly on the front buffer. The main problem is that modern GPU rendering is “pipelined”, so it's entirely possible to have a drawing operation that takes 16ms and a compositin…

This assumes you are synchronizing the updates with the monitor's refresh cycle, however if you aren't (and the major reason you see lag in compositors is because they do such synchronizations) then composition is indeed slower since it involves several more moving parts and the need to orchestrate the refresh of multiple windows (as opposed to the instant "i want to draw on the screen now" model that X11/Xorg without a compositor and Windows without DWM use).

I give a few more details here:

https://news.ycombinator.com/item?id=15748880

Re: Desktop compositing latency is real

#129

Post by nickjj »

swah wrote: post #117
nickjj wrote: post #23

This is also why picking a good monitor is important for software development. Some monitors have tons of input lag (60-70ms) and that's the time it takes for you to see what you're typing to reach the display. This also includes the time it takes to see you move your mouse cursor too. I did a huge write up on picking a good monitor for development which can be found at: https://nickjanetakis.com/blog/how-to-pick-a-g…

Is that still a good monitor vs 4k Dell offerings? Reasonable price here in Brazil.

Yes. I'm going to buy a second one in a few days (waiting to see if there's any Black Friday deals).

At this point I've been using 1 for almost a year and it's been nothing but great. I want to get a second one and orient it vertically.

Re: Desktop compositing latency is real

#130

Post by emn13 »

I will gladly believe this is a real problem, but this page does not demonstrate that (at least not convincingly); the metric used is simply too poor.

To quote:

> I used my own hacky measurement program written in C++ that sends a keypress to the application to be benchmarked. Then it waits until the character appears on screen. Virtual keypresses were sent with WinAPIs SendInput and pixels copied off screen with BitBlt.

So, this is measuring some rather artificial and fairly uninteresting time.

You really do need to measure the complete stack here, especially if your theory is that issues like vsync are at stake, because the level at which vsync happens can vary, and because there are interactions here that may matter. E.g. if there are 100 reasons to wait for vsync, and you remove one of them... you're still going to wait for vsync. It's not 100% clear that this measurement actually corresponds to anything real. Also, note that a compositor need not necessarily do anything on the CPU, so by trying read back the composited image, you may inadvertently be triggering some kind of unnecessary (or OS-dependent sync). E.g. it's conceivable that regardless of tearing on screen you want the read-back to work "transactionally", so you might imagine that a read requires additional synchronization that mere rendering might not.

And of course: all this is really complicated, there are many moving part's we're bound to overlook. It's just common sense to try to measure something that matters, to avoid whole classes of systemic error.

Ideally, you'd measure from real keypress upto visible light; but at the very least you'd want to measure from some software signal such as SendInput up to an hdmi output (...and using the same hardware and as similar as possible software). Because at the very least that captures the whole output stack, which he's interested in.

Another advantage of a whole-stack measurement is that it puts things into perspective: say the additional latency is 8ms; then it's probably relevant to know at least roughly how large the latency overall is.

Post reply on HN