They are, somewhat. You can theoretically make a safety factor 1.0 if you're really absolutely certain of the input criteria.
Safety factor is really just a (somewhat blunt) method of managing risk. It's just admitting that we don't know the true load spectra that a design is exposed to, so we take what we believe is the max load and then slap a multiplier on it to manage how much uncertainty we expect, be it from estimates of that load, or dynamic factors, etc. etc.
We then also consider the consequence (since risk = likelihood x consequence). If the consequence of a failure is that our balsa-wood model bridge falls over, we shrug and keep it low. If it's that our pressure vessel undergoese a BLEVE [1] in the middle of a population centre then you jack it right the hell up.
There's nothing that specifically requires a safety factor. We could spend millions of dollars and thousands of hours understanding exactly the load spectra a design experiences, but that may be prohibitively expensive in the case of designing a bridge, so we instead accommodate more risk by overdesigning the item. In a space application where every kg of launch mass represents big $$$, then spending that extra time and money to understand the load specifics in more detail makes sense.
[1]: https://en.wikipedia.org/wiki/BLEVE