Well, so I can only tell you as much as I know and understand. Some of this pulls in some outdated information too.
So, JVMs and languages that abstract the underlying machine are always going to have overhead. The original interpreted stack-based JVM model is really bad for performance because you can't do great optimizations on the code because you can't have a great view of the operands that are being defined and then subsequently used, on top of that you have to either JIT or interpret code which also has overhead. This is why Android's original Dalvik VM originally started by converting the Sun byte code format to a register based format. So, now you have a format you can do some optimizations on: great. But you still depend on a VM to generate and optimize for native code: that means code-caches and that means using excess memory to store the fast optimized code you want to run (which could have been evicted, so more overhead when you have to regenerate). Next you have frameworks like the classic Swing in Java that were frankly implemented with priorities that did not include having a really great and responsive experience even though its platform agnostic as far as the way it draws widgets. These days we can take GPUs for granted to make this approach work, but a lot of the Java UI stuff came from another era.
I am not really sure if I am right here, but to me all this means that to have made the Java system work well for modern PCs and mobile it would have required a ton of investment. As it turns out, a lot of that investment went into the web and android instead of polishing Sun and Oracle's uh... product.
Java's also kinda been sidelined because for years Oracle threatened to sue anyone that dared fork it as Google had, and Microsoft kinda spent a decade making C# and .NET more confusing than it already was so theres that too.