Earlier quoted context omitted.
You're wrong about how services that cost 9+ figures to run annually are budgeted. 1% CPU is absolutely massive and well measured and accounted for in these systems.
So you prematurely dump hardware you already own when you see CPU usage go down? I don't think so.
For individual services what that means is that for something like Google Search there will be dozens of projects in the hopper that aren't being worked on because there's just not enough hardware to supply the feature (for example something may have been tested already at small scale and found to be good SEO ranking wise but compute expensive). So a team that is able to save 1% CPU can directly repurpose that saved capacity and fund another project. There's whole systems in place for formally claiming CPU savings and clawing back those savings to fund new efforts.