A lot of it wasn't so much design inexperience as an intentional difference in design-style due to those factors B and C.
Where today we speak of "replay value", of games that you finish and then want to play again, no such concept existed back then. Instead, it was assumed that a game that was "beaten" (let alone "mastered") would be put away, and so a game should resist being beaten for as long as possible to make it "more worth the money."
This was partially the quarter-eating arcade-game philosophy leaking into home-console gaming, and partially the knowledge that parents want the $50 they spend on a distraction for their kids to stretch as long as possible.
In the 80s, since games were size-limited (small tapes/cartridges and low memory headroom), this translated to games that were preternaturally challenging to stretch out their play-time. This is the classic "Nintendo Hard" game-design philosophy; but it's also responsible for grind-fest RPGs (another way of taking a small amount of content and padding it out.)
In the 90s, with lower size-limits, we instead saw the temporary emergence and flourishing of the really long game—everything was advertised by talking about how expansive the world was, and how many hours of content there were to get through. We still look back on this era as being responsible for the creation of many classic action-adventure and adventure-RPG games for this reason.
All this stopped when we hit the 3D era and art-asset production became the overriding cost of producing a game. Games weren't limited in "content" in the scenario/script sense, but they were limited in "content" in the unique-things-to-see sense. There was a temporary burst of interest in exploiting this divide directly (some early PSX games fiddled with procedural generation to overcome the problem), but this was limited to a few minor studios; the majority instead moved their games toward being increasingly high-production-value cinematic affairs.
With a fixed amount of stuff to see, getting the player to care about looking at it all more than once became the overriding concern; thus, replayability rather than difficulty became the goal. (Replay-value was always part of the design philosophy of some studios, but now it became part of the basic culture of the industry.) This is when you start to see things like "missions" and "achievements" emerge, that give you many things to do within the same few set-piece areas.