This is why I use SMS as my second factor for my Google account. Much harder to lose. It could be vulnerable to sim swapping attacks, but I consider Google locking me out of my own account a more likely threat (and frankly I'm probably not a high-profile enough target for anyone to bother with that, and in any case they'd still need my password).
Instead of SMS, get a pair of yubikey recommended by some other posters, so you are not depending on your mobile provider as they own the number and it is just "rented" to you.
I think you can also do that with an iPhone and the Google Smart Lock app.