Earlier quoted context omitted.
I've always had lower end hardware than my peers (still do, except at work!) so maybe you had a different experience than I did... But here goes my experience: Emulators are much harder to play than the original console because milliseconds matter! Maybe not so much for puzzle games (like the Zelda franchise)[0] but certainly for action titles... I don't know much about this "RetroPie", but it sounds like a raspberry…
The main disconnect is how much buffering/pipelining there is of inputs and outputs on modern platforms. This affects native games just as much as emulators, but most people don't complain because they're used to it. Modern systems driving LCD TVs routinely hit total input latency of 5+ frames. Oddly enough, we might get the necessary stack enhancements for lower input latency thanks to VR development, since in VR it…
Found the source of 90~% of the latency.