Earlier quoted context omitted.
Having worked on the particular issue of streaming real-time video over wireless, I can say that the main issue is not link latency. The main lag comes from encoding/decoding. If you do it naively you encode frame-per-frame (encoding slices is more difficult), and the encoder does not only outputs iframes: you get partial frames that depend on both previous and future frames. Also the decoder does not always output f…
What about scrapping the conversion to streaming video entirely and replacing it with a networked graphics protocol that allows one PC to draw on another's graphics natively?
I mean, a 720p60Hz stream in 4:2:0 (12 bits per pixel) still amounts to 663Mbits/s. You won't get that out of a gigabit link realistically (at least not over IP). Of course you could use a lightweight compression algorithm, but you'll have to divide this bandwidth by at least 5 to make it manageable for the average home network I'd say (I have absolutely nothing to back that last number, but 100Mbits/s doesn't look too scary...). And that's only for 720p remember.
I think if you plan to stream HD video over the network you have to encode and be clever about it.