It depends as much on the genre as it does the game. Some games aren't intended to be immersive. Take Action Fighter, The Ninja or Sonic on the Master System -- all hugely awesome action games but none are trying to build suspense-based drama, which is where the feeling of immersion is really felt the most. Like the catacombs in Phantasy Star. All of the aforementioned are Sega Master System titles but action titles invoke very different types of concentration to the latter game.
Another example is Sultan's Maze on the Amstrad CPC 464 (bottom of http://www.cpcgamereviews.com/s/index26.html). It's an 8bit game for a 64KB machine and yet that ghost used to make me jump every time despite being rather questionable graphics even for the 80s.
Imagination is definitely powerful enough to fill in the blanks but you need a particularly kind of game to draw you in.
Let's also not forget that as we age, our imagination becomes less powerful. People often blame our fondness of retro games as sample bias (like you have done too) but often it's just younger gamers having better imaginations and a lower bar for what they find entertaining. I've seen this first hand playing Atari 2600 games with my kids. Playing Jungle Hunt (https://www.youtube.com/watch?v=BZxw4MfXnNs) my, then, 4 year old son corrected me when I referred to what appeared to me as a blocky shark -- to which my son replied "It's a crocodile daddy". And it obviously is -- but that I mean it is only obvious to adult me now that I've been told.
I've played modern generation games, grew up with 80s-era BASIC micro computers and have played on practically every system -- console and computer -- released in between. And as you might have guessed from my stories above, I still own and collect these systems too. But what made a game immersive wasn't the graphics nor voice acting (though these elements can certainly pull you from the moment if they're too jarring), it was games that required one to concentrate on longer term strategies from perspective of avatar rather than concentrate on quick reactions from immediate threats (as one typically sees in arcade-style action games).