Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Most UI applications are broken real-time applications

thelig.ht

Post reply on HN
Print view View original

21–30 of 151 posts

Re: Most UI applications are broken real-time applications

#21

Post by josephg »

dllthomas wrote: post #19
masfuerte wrote: post #12

Earlier quoted context omitted.

Not just that. You hit a key and the game could react in literally the next frame. There was an immediacy you don't get now.

Metal Slug on the PS4 was jarringly unresponsive compared to my expectations, though I'm not sure I've played it on an arcade machine in 20 years so maybe it's my memory or physiology at least partly to blame.

Strangely enough, I had the same experience with diablo 4 until I turned down some of the graphics settings. I'm not sure if it was some DLSS / frame generation thing or what, but the game felt sloppy to me. Especially in comparison to something like diablo 2 which feels razor sharp.

I wish modern game developers & graphics pipelines would spend more effort optimizing input latency. I'll stop noticing the graphics 2 minutes into playing the game. But if the input is laggy, I'll feel that the entire time I'm playing. And the feeling of everything being a bit laggy will linger long after I stop playing.

Re: Most UI applications are broken real-time applications

#22

Post by mrob »

TacticalCoder wrote: post #5
smokel wrote: post #2

I like this perspective. In the demoscene days, we used to obsess over 60 Hz guaranteed framerates. Code was constantly being profiled using raster bars. Our effects might have been crappy, but all was buttery smooth. Then someone decided it would be a good idea to create a common user interface that would work on many different framerates and resolutions, and all was lost. Or people would try for complex 3D graphics…

> Code was constantly being profiled using raster bars. For those wondering: we'd change the background color from, say, black to blue for the "physics" part, to green for the "drawing" part, to purple for the "audio rendering" part... All in the same frame, while the frame was being drawn. So you'd see on the border of your screen (usually outside of the drawing area you'd have access to) approximately which percent…

A lot of that smoothness was the lack of sample-and-hold blur on CRTs. Gaming LCDs now have modes that strobe the backlight to simulate this effect. Additionally, the importance of sample-and-hold blur declines as the frame rate increases. A 360Hz LCD displaying 360fps motion can look reasonably sharp even without strobing.

Blur Busters has detailed explanations, e.g.:

https://blurbusters.com/blur-busters-law-amazing-journey-to-...

Re: Most UI applications are broken real-time applications

#23

Post by gpm »

PaulKeeble wrote: post #18

You do not want real time systems design for a GUI, its a lot more work and it wastes the vast majority of processing power to guarantee performance. You just need to be a careful about the amount of work you do in event handlers and split off heavy or IO work and split big updates. The problem is all concurrency is a lot harder than serial and so it tends to be something that is delayed until its really necessary so…

You don't want to do the design work, but you do want the results.

Maybe some sort of magical framework will figure out how to get the latter without the work in the future.

Re: Most UI applications are broken real-time applications

#24

Post by scott_w »

dllthomas wrote: post #19
masfuerte wrote: post #12

Earlier quoted context omitted.

Not just that. You hit a key and the game could react in literally the next frame. There was an immediacy you don't get now.

Metal Slug on the PS4 was jarringly unresponsive compared to my expectations, though I'm not sure I've played it on an arcade machine in 20 years so maybe it's my memory or physiology at least partly to blame.

If I recall correctly, the PS4 controller has a disgusting amount of input delay. I noticed this most strongly on the FFX Remaster where you need to hit timer events before the arrow enters the shaded box.

It doubly screws you in the chocobo racing section because the controls assume no delay, meaning you can’t respond to hazards in time and constantly overcorrect movement.

Re: Most UI applications are broken real-time applications

#25

Post by kridsdale3 »

valbaca wrote: post #8
m463 wrote: post #3

I believe I agree. The iPhone UI must have either a realtime UI (or something pretty snappy). It responds to finger inputs in a bounded way.

Dispatch queue. Android has the equivalent thing.

More than that even. Since iOS 1, the majority of compute-intense work after a touch event has happened in an entirely different process, which has the highest OS-scheduler priority. This is what CoreAnimation is based on. Unlike a game engine or other "realtime" app, the OS itself is doing everything possible including preempting your main-thread to ensure that anything animating takes priority.

Re: Most UI applications are broken real-time applications

#26

Post by mrob »

dllthomas wrote: post #19
masfuerte wrote: post #12

Earlier quoted context omitted.

Not just that. You hit a key and the game could react in literally the next frame. There was an immediacy you don't get now.

Metal Slug on the PS4 was jarringly unresponsive compared to my expectations, though I'm not sure I've played it on an arcade machine in 20 years so maybe it's my memory or physiology at least partly to blame.

The Metal Slug games are already at a disadvantage because they run at 30fps, unusually low for arcade games. All else being equal, lower frame rate gives higher input latency.

Re: Most UI applications are broken real-time applications

#27

Post by orbital-decay »

TacticalCoder wrote: post #5
smokel wrote: post #2

I like this perspective. In the demoscene days, we used to obsess over 60 Hz guaranteed framerates. Code was constantly being profiled using raster bars. Our effects might have been crappy, but all was buttery smooth. Then someone decided it would be a good idea to create a common user interface that would work on many different framerates and resolutions, and all was lost. Or people would try for complex 3D graphics…

> Code was constantly being profiled using raster bars. For those wondering: we'd change the background color from, say, black to blue for the "physics" part, to green for the "drawing" part, to purple for the "audio rendering" part... All in the same frame, while the frame was being drawn. So you'd see on the border of your screen (usually outside of the drawing area you'd have access to) approximately which percent…

> People don't understand that their 144 Hz monitor running a 3D game still doesn't convey that smoothness that my vintage arcade cab does when running an old 2D game with pixel-perfect scrolling while never skipping a frame.

The actual motion-to-photon delay (measurable with a high-speed camera) in a well optimized competitive FPS is somewhere around 20ish ms or less these days, so it's on par with it in regards to the input lag, and overall it's far smoother because of the higher frame rate, better displays, and way more thought put into tiny nuances competitive players complain about. A proper setup feels extremely smooth, responsive, and immediate.

Re: Most UI applications are broken real-time applications

#28

Post by dzaima »

The reality is that optimizing for the common scenario just gives such significant benefits that the worst-case scenario becomes practically infeasible to even consider.

And it's not just OSes that don't particularly care about real-time - modern processors don't either. e.g., in the common scenarios it's possible to load maybe eight 8-byte values per nanosecond (maybe eight 32-byte values if you're doing SIMD), but if they're out of cache, it could take hundreds of nanoseconds per byte. Many branches will be predicted and thus not cost affect latency or throughput at all, but some will be mispredicted & delay execution by a dozen or so nanoseconds. On older processors, a float add could take magnitudes more time if it hit subnormals.

If you managed to figure out the worst-case timings of everything on modern processors, you'd end up at pre-2000s speeds.

And virtual memory isn't even the worst OS-side thing - disk speed is, like, somewhat bounded. But the number of processes running in parallel to your app is not, so if there are a thousand processes that want 100% of all CPU cores, your UI app will necessarily be able to utilize only 0.1% of the CPU. [edit note: the original article has been edited to have a "Real-time Scheduling" section, but didn't originally]

So, to get real-time UIs you'd need to: 1. revert CPUs to pre-2000s speeds; 2. write the apps to target such; 3. disable ability to run multiple applications at the same time. Noone's gonna use that.

Re: Most UI applications are broken real-time applications

#29

Post by _y5hn »

seanalltogether wrote: post #20

Even if you rebuilt the the entire OS and libraries on a separation of sync vs async code, you still come to the inevitable problem of propagating delays to the user in a predictable manner. So I've pressed a button to load a file that's usually really fast and this time nothing happens because the async call is taking it's time. Is that being represented to the user in some meaningful way? Is the button disabled unt…

UI was pretty much solved in the 90's including the problem of instant feedback. Then abandoned for shiny stuff.

Personally I set up i3 to open most windows asynchronously, so my flow isn't interrupted. It's great, but takes a bit getting used to windows not randomly stealing focus. It's not for everyone though.

Re: Most UI applications are broken real-time applications

#30

Post by anonymoushn »

dzaima wrote: post #28

The reality is that optimizing for the common scenario just gives such significant benefits that the worst-case scenario becomes practically infeasible to even consider. And it's not just OSes that don't particularly care about real-time - modern processors don't either. e.g., in the common scenarios it's possible to load maybe eight 8-byte values per nanosecond (maybe eight 32-byte values if you're doing SIMD), but…

I feel like this comment is from an alternate reality where the Windows lock screen doesn't just drop all your inputs for a random period of at least a few seconds when you try to enter your password.
Post reply on HN