I am not an expert but I will take a shot. CPU speed is in part determined by the clock. Faster clock, more data processed per second. That's the simplest understanding. Now for modern CPUs, we use a pipeline architecture. So just like how a compiler like LLVM has a tons of optimizations, the same can be said for CPUs. Your CPU doesn't quite execute instructions one by one like a hardware Turing machine. It consumes a large chunk of instructions, try to figure out what's needed and what's not, and then process it. This is all done the few pico seconds of each cycle. It's less like a car manufacturing line and more like a fruit sorting machine, different operations gets executed simultaneously if the processor can prove that they are not e.g. required to be in order. A CPU with more transistors can easily process more optimizations i.e. bigger pipeline. You can also throw in other stuff like caches to squeeze out even more performance. The other side of Moore's law is economics, essentially cost of computing gets halved, more processing power for half the cost.
Same chip, more power. Now for CPUs, clock speed has pretty much plateaued in the last decade. It is increasing, but slowly. At the same time, there are limits to pipelining, and if you get too eager with optimizations, you end up introducing vulnerabilities (since this is HN, I think I don't need to cite examples, there have been plenty). We are approaching the limit of transistor scaling and the clock isn't magically oscillating faster so there is now greater urgency on the architecture side. Current software is unable to take full advantage of multi-core processors or parallelism to the extent of making Moore's law great again. (the last time Intel tried to launch a CPU with emphasis on parallelism didn't really work out, look up Itanium). It shifts a lot of work to the compiler end and makes life unnecessarily difficult for compiler authors.
Now GPUs are designed for running multiple computations simultaneously, like CPUs, there's the fact that most operations cannot be trivially parallized. Matrices and linear algebra can because of obvious reasons — large grid of numbers, easier to process at once than with a for-loop. Most problems are not like this. The next step for computing may be FPGAs, digital circuits that can be reconfigured on-the-fly. Certain new Intel CPUs have this and most processors have a basic version in the form of microcode. Whether this will work out, I do not know. FPGAs is too more compatible with trivially parallized tasks than your usual CPU based workload. Theoretically in future you can perhaps spend 20min compiling Minecraft or Factorio to FPGA bitcode during the installation process, though more likely it is only the processor intensive bits that gets optimized. When you load the game, part of the chip gets reconfigured for it. I believe this will benefit long running processes more than short ones.
Now I am no subject domain expert so somebody with more insider experience should elaborate on this.