the thing about 'adding a phone number' is that hijacking somebody's DID is fairly trivial these days for a good social engineer, you get the customer service department at somebody's cellular carrier to port out the number, or activate it on a new SIM card put into a burner. the SS7/PSTN is horribly broken. SMS based "2FA" is not actual 2FA
I think "trivial" is generous. For context, the current market rates for a SIM swap ranges from several thousand USD (T-Mobile) to well over fifty thousand USD (Verizon). It is not really something most people should lose sleep over, in my opinion.
Working in the telecom industry I've seen the pressures that first tier phone service reps are under and how they can be socially engineered, if someone is in possession of enough pieces of a person's identity already, to issue a new SIM or port out a number.