A few suggestions: 1) Call your cellphone carrier and ask to set up a password/PIN to be used for when you call into the customer service phone number. 2) Consider your phone number and SIM card insecure. The phone carriers are ignoring the SIM swap problem even though they know how much damage it's causing. Give your phone number to as few companies as possible. Phone services such as Google Voice work without a SIM…
This is what I'm worried about, if I don't have a phone number on a gmail account, won't it make it harder to get access to that gmail account if I am locked out. Even though there's obviously a bad security loophole with just having a phone number on your account as a recovery option, is it not worse to be locked out of your account with no way to get in?