I have been saying this for years.
The amount of knowledge one needs to port a phone number is unbelievably little, and peoples very nature to be helpful works against us. Up until maybe just very recently you needed the account number of the phone number and the last four of a social security number... sometimes, just the account number. Also, the last four of one's social security number is perhaps the shittiest way to authenticate _ANYTHING_. For many years, a lot of sites online would show you the last four of an account holder's SSN (and some places still probably do) if you have an email address, correct name, and phone number or physical address.
Getting the account number would likely be even easier thanks to helpful store reps... Just go in and make up an excuse why you need it or forgot it, it's like "social engineering 101" because it seems so benign to most people. You already know the name, address, and phone number-- you just "forgot" your id at home... Or one could just listen to them call each-other write down their info and then call another store.
With those two things in hand, the phone number is pretty much the attacker's, and getting it back would take more than enough time for extensive amounts of damage... ESPECIALLY if that is the only line on the account (or they took all the lines)... I'd guess a bare minimum with near immediate recognition of the real problem (your number heisted) and police involvement, probably a minimum of ~12 hours.
So, if phone numbers are so bad why are they ever used? IMHO, that's because they aren't to provide security, they're to provide easy tracking between your virtual life and your physical one. You're only securing the businesses data pipeline, not your personal data.
If you want 2FA (and everyone should) use Google Authenticator or a Yubikey... or whatever I'm not trying to shill brands just ideas that work.