I don't think the reasons are too complicated here, I think they really just come down to a mixture of the following:
1. Most games are created as interactive experiences, and the storytelling reflects that. For example, look at something like the Metroid Prime series. Almost all the lore is found in logs and in universe worldbuilding, not in the form of a linear narrative. That's hard to adapt into a non interactive medium, because you have tell such a story in a completely different way.
2. Many successful games either have poorly thought out or non existent stories because they focus on the mechanics and gameplay. See for example the Mario platformers (the RPGs might work better here), the Pokemon series, Sonic, any puzzle game like Tetris, Minecraft etc.
(It's also worth pointing out that much of what's needed for a fun game would be seen as bad writing in any other medium, since a realistic hero likely wouldn't be able to handle the challenges in many games. Most video game protagonists would likely be seen as Mary Sues/wish fulfilment in a non interactive medium).
3. Hollywood has a tendency of going after popular franchises regardless of whether they have a story that works in film form. This means being stuck trying to adapt games that simply don't work well in the format rather than ones like adventure games or visual novels which would adapt pretty well. The recently announced Tetris movie trilogy is a good example of this, since the source material has basically nothing you can adapt into a non interactive film.
Video games are just difficult to adapt by virtue of the medium and what's needed to succeed in it.