The difference with sim/4X/any genre that's fallen outside of the general "popular games" trend is that they ended up becoming overly specialized and inaccessible to newcomers.
Yes, technically you can still play a bunch of cool 4X games, but most people still making them (with the exception of like, CAs Total War, which also notably defocused the 4X part in favor of RTS combat) is copying the Paradox approach and Paradox is largely known for its absurd depth and mechanics... which unfortunately translates to the games devolving into spreadsheet simulators with fancy graphics.
Civilization was the right mix between "flavorful civilizations", "fun endgoals" and "reasonable to achieve victory against the computer". Most 4X games, because they copied the Paradox approach, kinda suffer from the fact that they pretty much demand immediate mastery of the systems if you want to succeed, unless you're playing with friends (in which case you have more control over each other's bumbling and one person can just hyperfocus on learning one system, while the other can focus on another.)
Its the difference between SimCity and Cities: Skylines. SimCity (before the final game) is a sandbox toy that lets you build a super dysfunctional city but y'know, as long as the taxes don't go in the red, you're doing fine! Cities: Skylines otoh has way harsher feedback tools if you don't build a highly functional city. That's fine if you're already tuned in on SimCity as a game and crave a higher challenge, but if you're new to these kinds of games, Cities: Skylines is just too much.