I've been gradually switching towards completely using IDEs for like 8 years now, and in the past 2 or 3 I've broken free completely. Some of my symptoms are the same as yours: Frustrating package management, obvious missing gaps in functionality.
My conclusion is slightly different: I don't need a text editor anymore. I need an IDE and it's a bonus if it's a good text editor.
Efficient text editing is nice. But I spend a lot of time jumping between files, looking up definitions, reading documentation, tracing through code, debugging, comparing tests and regular code, etc. Plus all of the features that you expect: (colorblind-friendly) syntax highlighting, syntax highlighting, linting, etc. This is where the package pain comes in. I don't want to figure out how to configure the cross product of "every single IDE feature that I need" and "every single language that I use." I gave up. It's too much. I want my environment to either do it by default, or install a single plugin per language and never think about it again.
I'm not sure how Emacs can get "more modern," because being modern would likely start from the premise that you'd need to allow the core innovation of IDEs. They provide a pluggable framework to get uniform behavior across every language that you use. That ship sailed in Emacs - how do you move away from 40 years of "we ship a vanilla experience and you customize your environment completely to your own liking"?