HSMs are shit. In a previous role we used a major vendor's HSM to protect our private keys. VERY expensive kit, more expensive than the load balancers and servers combined. We needed to use Elliptic Curve keys for a particular customer - so it got even more expensive as we had to buy: 1. A license from the LB vendor to use the HSM 2. A licence from the HSM vendor to use EC with the LB. ... even though they trumpeted…
Is there any way to do what a HSM does, in software? (Maybe with OS support)
In contrast, while you could think of, say, an ssh key agent as a sort of "software security module", such things are inherently vulnerable to low-level attacks that side-step the intended interface. Say, a kernel exploit that grabs the decrypted key straight from the agent's memory.
So if you need to keep your secrets safe even if you unwittingly use them on a comprised OS, you can only pull that off with a hardware module.