Earlier quoted context omitted.
Check your settings on your call that it is on Voice Isolation; there are three settings IIRC that tell the microphone what to focus on.
Oh wow, this is really helpful. I did a quick test and this seems to make a big difference! For anyone reading: if you're on a call, pull down on control center and there's a Mic Mode setting that appears that you can change to "Voice Isolation". This setting won't be there if you're not in a call. Here's an article that explains it: https://www.lifewire.com/use-voice-isolation-on-ios-15-52064...
I wish Voice Isolation could be pre-trained on our voices, similar to how Face ID pre-trains on your face. That way, the phone's software can know to recognize your voice and not cancel those out.