TLDR: Telcos really are the weakest link, and you should not rely on your mobile phone number for 2FA.
Background: I have worked in IT Security at an Australian bank, and had close ties to the Internet Fraud department to help them understand fraudster's tactics.
Many banks use SMS for 2FA. Australia has a law regarding how long it should take customers to switching telco providers (called 'Porting' because your retain your phone number), and the timeframe in which this must be completed (90% within 3 hours, 99% within 2 business days). If the Telco doesn't complete in this time period, you can raise a complaint to the Telecommunications Industry Ombudsman.
Example: If you are currently with Telco A, to port your number to another company, you call Telco B and provide your details. They take care of the porting process, and you can have your service running on a new phone and SIM within 3 hours.
"All you need to have with you is your mobile number, the name of your old mobile provider, your account type (pre- or post-paid) and your account number. We'll handle the porting process from there. It can take from three hours to three days, but we try to do it as fast as we can."
Source: https://www.cnet.com/au/news/switching-telcos-easier-than-yo..., 2012
To make matters worse, the fraudsters would then change the details at the new Telco B (i.e. my address is now 123 Rainbow Road, and my mother's maiden name is Smith, not Jones). When the victim called Telco B, when Telco A told them a porting request had been completed, they'd say "Sorry, we have no idea who you are and the details you're providing don't match our records". It can take days to sort the whole thing out, by which time, your Internet Banking has been compromised and funds transferred out.
This was a major problem for Australian banks, because they cover the losses for customers if you lose funds as a result of Internet Banking, as long as you weren't negligent (e.g. you left your Internet Banking logged in on a public computer in a library, or something).
If you are relying on your telephone number as a security mechanism, I would change to something else. Something you have, ideally (Google Authenticator, a physical hard token, etc.).
Sources:
ACMA Porting Rules for Telcos: http://www.acma.gov.au/Industry/Telco/Numbering/Portability/...
Example A: http://lifestrategies.net.au/wp-content/uploads/2015/03/Marc...
Example B: http://www.itnews.com.au/news/45k-stolen-in-phone-porting-sc...
Example C: http://www.news.com.au/finance/business/banking/customer-sca...