The main difference between games then and now, I feel, is that they didn't have the crutch of excellent graphics to fall back on to wow people, they had to make their games fun and worry about the nonsensical story lines later. Gameplay distilled down into its most pure form. A lot of these games are extremely difficult and require twitch reflexes, which is something a game burdened with pushing a lot of pixels has…
It's not that they have trouble replicating it, they are intentionally avoiding it. With higher fidelity comes higher development costs, so they have to sell more copies to make a profit. One way to do that is to make games easier. People in general want games that can be completed with the least amount of effort on their part.