I think I finally figured part of this out. Like you said, employers treat workers as commodities. This worked out great for them in the short term, as they saved money by not giving raises etc. In the long term, though, workers started realizing they often had to go back to the market to get meaningful pay raises. Now you have a situation where workers are switching as often as every 12-24 months, and the thing not previously accounted for is onboarding costs. In a situation where workers stay an average of 5 years, let's say they take a year to become net productive, you're losing about 20% of their time as an investment for 80% payoff. Now, though, that static year of onboarding costs accounts for 50-65% of their entire tenure. That's a huge swing against companies, who are suddenly realizing retention might be a problem. So the economic/management theory that everyone is just an interchangeable commodity is running up against the wall of reality, where second-order effects have taken over and destroyed a bunch of business value that could be captured in a more stable environment.
There's an additional problem that firms are caught in a game theoretic trap. If you're the only firm providing stability, your investments in that in the short term might make you less competitive on pay, which means you'll train up people who might leave seeing higher pay elsewhere. So stability investments might be rational in the long term, but because of market conditions you get killed in the short term, and you end up tempted to abandon your strategy after a few bad quarters where it looks like they aren't paying off.