Start with the Normal or Multivariate normal. We start with this because it is the distribution that emerges naturally when we have lots of little errors adding up around a basically consistent effect (see also [0]). So basically any situation where something ought to happen but 101 things could also happen to cause small changes will turn out to be normally distributed. An exceedingly common occurrence + one that almost inevitably turns up when dealing with a practical error prone process.
Now that we have a normal distribution, the mean is an obvious metric to pick because it captures the notion of 'middle' in a useful sense. Then we have proofs that we can characterise the normal with the mean and 1 other parameter (normal normal) or a matrix (multivariate normal). We call that the standard deviation^.
The exact formula wasn't a coincidence. The normal can obviously be characterised by the mean and a statistic from another formula. There were a bunch of experiments tried (eg, using |x| instead of sqrt[x^2]) but it turned out that sqrt[x^2] had some other nice property that minimised some sort of error so they went with it as a standard. I forget what one, might be error of estimating the true parameters from a sample or similar.
We could characterise the the normal as an infinite sum or something quirky, but when people say 'easy to work with' they mean instead of a function or something quirky we can simply pick a number.
Standard Deviation isn't as important when working with non-normal distributions, although I think it still turns out to be useful. But its importance is that it characterises a normal apart from the information captured in the mean. I'm not a mathematician, YMMV, could be wrong, standard disclaimers.
[0] https://en.wikipedia.org/wiki/Central_limit_theorem
^ I'm not going to edit this but it occurs to me that we call it the Variance. Same thing as std. dev in my opinion.
' There is a fairly subtle observation to make - Normal is characterised by mean and std. dev, but the most efficient ( https://en.wikipedia.org/wiki/Efficiency_(statistics) ) unbiased estimator of the std. dev of the population is the adjusted std. dev of the sample. Therefore, accounting for mean, you can't get a more efficient characterisation of the normal than mean & std. dev. Ie, if you picked a formula other than std. dev then the most efficient estimators to characterise it would still be mean and std. dev. Don't recall if there are equally efficient choices but I think that proves there are none better.
That might have been the logic for why std. dev was chosen. Just a guess.