Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack: Latency Mitigation Strategies

altdevblogaday.com

Post reply on HN
Print view View original

51–60 of 106 posts

Re: John Carmack: Latency Mitigation Strategies

#51

Post by stcredzero »

laureny wrote: post #45
stcredzero wrote: post #36

Earlier quoted context omitted.

> I wish people would start treating text editors that they are developing this way. IDEs. Most of the things people do in modern IDEs were done in Smalltalk (albeit, with uglier interfaces and some manual steps) over a decade back. One of the differences, though, is that these things were mostly highly responsive, because they were bloat-free. (Can't speak for IBM VisualAge. Also, one could rightly complain that the…

> One of the differences, though, is that these things were mostly highly responsive, because they were bloat-free. That's not my recollection of the Smalltalk IDE at all. I especially remember how selecting was crazy slow: press down the button, move the mouse to the end of the line and the selection takes two seconds to catch up with the cursor.

> That's not my recollection of the Smalltalk IDE at all.

Which one and from what time period? My experience was mostly with VisualWorks. Even just with that, there are variations. Going from version 3.* to 5i was actually a step down because of hastily released stuff from a company in turmoil. Also, lots of open source/free implementations started behind the curve in terms of technology. (Still doing bytecode interpretation instead of JIT.)

Re: John Carmack: Latency Mitigation Strategies

#53

Post by bbatchelder »

Scene_Cast2 wrote: post #40
breckinloggins wrote: post #31

It sounds like we would benefit from GPU and display manufacturers providing a set of standard low-level control primitives so that people like Carmack who know what they are doing can really play around with the entire pipeline without having to worry about all the things the cards and the displays are doing behind their backs. For example, a GPU could have a set of standard settings with full buffering and all the…

Regarding your last point: it's getting better. Modern TVs have "game mode", where latency is minimized. The latest LCD panels use embedded Display Port, eDP. The latest eDP standard introduces a frame buffer on the LCD itself, where you can just write deltas and tell it to "swap". I think this is quite similar to what you're proposing, in concept.

You may already know this, but the "Game Mode" on modern TVs is just turning off the 120Hz/240Hz interpolation that is done to "improve" the picture (or is required for certain 3D systems to work).

The interpolation introduces significant latency that its obvious (and frustrating) during gaming.

I actually run my TV in game mode by default, because the interpolation done in 120/240Hz mode makes everything look like it is slightly unreal and shot on video. Definitely uncanny valley territory.

Re: John Carmack: Latency Mitigation Strategies

#54

Post by stcredzero »

purplelobster wrote: post #32
untog wrote: post #23

Earlier quoted context omitted.

But at that point you've gone from a games console being something that takes up a small amount of space underneath your TV to something that takes up an entire room. From requiring you to pick up a controller to putting on an entire outfit. I'm just not sure the average consumer is in any way ready/willing for that.

I think the first successful VR you'll see will be closer to an arcade machine than a home console.

> I think the first successful VR you'll see will be closer to an arcade machine than a home console.

I was just thinking about this. My development rig has 2 27" screens. If I added a 3rd one, this would cover most of my field of view horizontally. What if someone sold a console rig packaged with 3 screen and a mount? (Work with IKEA to produce a compatible cheap desk.) This kind of thing is already done and is awesome for vehicular games. (Racing, tank, aircraft...) A 3 screen rig also could still strike a kid as "awesome," so would draw money from the wallets of parents.

Or, for arcade-type settings, how about a seat with a projected 360 dome, HOTAS controls, and motion simulation?

Re: John Carmack: Latency Mitigation Strategies

#55

Post by stcredzero »

knodi wrote: post #41
stcredzero wrote: post #37

> Updating the imagery in a head mounted display (HMD) based on a head tracking sensor is a subtly different challenge than most human / computer interactions. Doesn't the military already have this solved for head mounted displays for attack helicopters and 4++ generation fighter jets? Heck, they have augmented reality displays for that matter. EDIT: Many of these problems could be solved by putting an entire purpos…

Yes, they have with very powerful hardware thats not very consumer wallet friendly.

> very powerful hardware thats not very consumer wallet friendly.

Yes, but the mobile landscape may well have changed this.

Re: John Carmack: Latency Mitigation Strategies

#56

Post by modeless »

neverm0re wrote: post #30
modeless wrote: post #19

Earlier quoted context omitted.

Eye tracking would be awesome but it's not required for good VR. I fail to see how "using your head as a dumb camera joystick" is a problem; that's the whole point of VR. Motion sickness is a real problem for some people (but not all). With low latency and thoughtful game design I think it can be mitigated. The bigger problem is the social acceptability of blocking your entire field of vision for long periods of time…

> I fail to see how "using your head as a dumb camera joystick" is a problem; that's the whole point of VR. Perhaps you should try playing some VR games for a while. I clocked quite a few hours playing the old Virtuality SU2000 games like Dactyl Nightmare and I've idly kept up with HMDs. The chief problem is that without eye tracking, it's incredibly UNFUN to use your head for movements that your eyes could otherwise…

it's incredibly UNFUN to use your head for movements that your eyes could otherwise have done for you.

That's only a problem if your FOV is too small. With a large enough FOV and a light display there's no reason why eye and head movements shouldn't work exactly as they do in real life. The Oculus Rift has twice the angular field of view of the SU2000 in both dimensions, for 4x the subtended angular area.

Edit: Oh I see, your complaint is about using the head tracking to control a game, e.g. by pointing a gun. Yes, I think that's a bad idea. Head tracking should only control the camera. All game interaction should happen through a controller. A motion controller like the Razer Hydra would probably work well for this.

if you move your head for any reason you can't maintain focus on objects

Again, only true for crappy hardware. With a good enough display, 120 FPS, and low latency, there's no reason why tracking moving objects shouldn't work just fine.

I think a lot of people got disillusioned with VR because there's a lot of crappy hardware out there. Even the expensive stuff is crap. I tried Canon's augmented reality system at SIGGRAPH last year and the latency and FOV were awful, despite the $120,000 cost. But it doesn't have to be that way, and the Oculus Rift is the proof.

Re: John Carmack: Latency Mitigation Strategies

#57

Post by modeless »

ChuckMcM wrote: post #27
modeless wrote: post #2

I think head-mounted-display virtual reality is a huge missed opportunity for next-gen consoles. They could have really done it right in a way that PCs can't yet, with special hardware support for low latency and stereo rendering, and a guaranteed large audience so game studios could justify applying their full budgets to AAA VR titles. It would be a huge differentiator at a time when consoles seem to be converging t…

I don't think they were a missed opportunity. In order for them to 'miss' they had to be 'possible' which they haven't been for a long time. Its only lately where various things have come together to get close. I have followed the 'head mounted display' technology closely since its early inception in the mid-90's as a means of providing military pilots with situational awareness. The first problem with head mounted d…

$2K/unit? That's a gigantic overestimate. High quality head tracking is practically free today thanks to advances in cameras and MEMS sensors; you don't need laser gyros. The Oculus Rift costs $300, and is almost good enough for a home console. The only major improvement needed is a better display, which needn't cost more if sourced in quantities large enough, which Sony or Microsoft could definitely do. $500 for a complete system with display, controller, and console is definitely achievable, and that's the price point the PS3 started at. VR is absolutely feasible today.

As I said further downthread, I think a lot of people got disillusioned with VR because there's a lot of crappy hardware out there. Even the expensive stuff is crap. I tried Canon's augmented reality system at SIGGRAPH last year and the latency and FOV were awful, despite the $120,000 cost. But it doesn't have to be that way, and the Oculus Rift is the proof.

Nintendo saw the possibility of doing something new with the Wii, using then-new technology (MEMS sensors, tiny low-power cameras) to make an old, lame, expensive concept (motion control) work for the mass market, and was rewarded handsomely for it. If someone had that foresight with VR they could be making a killing right now.

Re: John Carmack: Latency Mitigation Strategies

#58

Post by neverm0re »

drivers99 wrote: post #49
neverm0re wrote: post #30

Earlier quoted context omitted.

> I fail to see how "using your head as a dumb camera joystick" is a problem; that's the whole point of VR. Perhaps you should try playing some VR games for a while. I clocked quite a few hours playing the old Virtuality SU2000 games like Dactyl Nightmare and I've idly kept up with HMDs. The chief problem is that without eye tracking, it's incredibly UNFUN to use your head for movements that your eyes could otherwise…

> The chief problem is that without eye tracking, it's incredibly UNFUN to use your head for movements that your eyes could otherwise have done for you. Mind you when the HMDs were much heavier back then it sucked a lot more, but it's still pretty shitty not being able to glance aside. Isn't that more of a problem of field of view? With a wide enough field of view, you could just look at whatever you wanted to. Basic…

Yes, you're absolutely right that a lot of what I'm complaining about would be solved with sufficient FOV and things certainly have improved since the SU2000 system in that regard. But we're not at 'sufficient' at the moment, at least in my experience based upon trying more recent examples, like an obscenely expensive Sensics kit. It's possible that it will be sufficient long before eye-tracking winds up in consumer HMDs.

But yes, the other part of my previous remark regarding 'dumb joystick' was more directly related to using the head for aiming/pointer duties, which also unfortunately has cropped up before and I quickly conflated the two issues together.

Re: John Carmack: Latency Mitigation Strategies

#59

Post by T-hawk »

breckinloggins wrote: post #39
efdee wrote: post #3

I feel stupid.

The view update part was the most confusing, but I tried to understand it this way: Imagine you had a magical game engine that rendered the entire world perfectly accurately for every point in space and direction a viewer could possibly be looking at. All you had to do was say: RenderGameFrameForEveryPossiblePoint(); ... // Who knows how much time viewerPosition = QuicklyGetViewerPosition(); TellTheGPUToShowWorldAcco…

That time warping has already been implemented in the project of Lagless MAME. It compensates for input and display lag by always rendering a few frames into the future. It's commonly used for games where frame-accurate timing is critical, notably 2D scrolling shmups and fighting games.

Lagless MAME renders into the future assuming that the state of the input controls remains constant over that future time, and saves the emulation state every frame. When a button is pressed or released or whatever, Lagless MAME rewinds to the saved state for that frame and quickly re-emulates from that point forward. So the result is to send your input back in time past the lag, to the moment in the emulation exactly synchronized to when you saw it on the screen. The experience isn't perfect -- your spaceship would jump a few pixels then move smoothly -- but by and large it's far superior to playing with the actual lag.

This technique could be used for lag compensation in almost any environment. The limiting factor is the cost of re-computing several frames of game state on every input action. Of course, as Carmack says, actually eliminating lag is far preferable to masking it with such techniques.

The lag compensation in Guitar Hero and Rock Band games works essentially this way too.

Re: John Carmack: Latency Mitigation Strategies

#60

Post by rmc »

breckinloggins wrote: post #31

It sounds like we would benefit from GPU and display manufacturers providing a set of standard low-level control primitives so that people like Carmack who know what they are doing can really play around with the entire pipeline without having to worry about all the things the cards and the displays are doing behind their backs. For example, a GPU could have a set of standard settings with full buffering and all the…

One problem is that lots of people think they know what they are doing, but actually don't.

You also have the problem of people choosing to use "expert mode" because it's faster, obviously.

Post reply on HN