My experience teaching is limited (but I have taught some, to be clear) but I have found learned helplessness to be the biggest barrier. People have varying aptitudes for different tasks, and varying aptitude and a finite lifespan does imply some people have a lower ceiling than others in a given subject, but humans are powerful general learners. They don't generally reach their ceiling in most subjects. The limit for someone "bad at math" is almost certainly self-fulfilling prophesies they internalized.
Speaking for myself I have in the last five years or so been learning I have much more of a capacity for making art than I had thought. My art is nothing special, but I am improving every time I practice. But when I was younger I thought that I was just good at STEM-yy things and bad at other things. Relatively speaking I am better at STEM-yy than art-yy things, and I'm probably worse at art-yy things than most other people. But I have huge room for growth and I think I will eventually produce some beautiful watercolors.
As an aside, I've also found that almost everyone thinks they're bad at math? My friends with PhDs don't think they're good at math but they've forgotten more than I know about it. I think I'm bad at math but I can prove a thing or two. My spouse thinks they're bad at math, and they can't do the things I can do. But a few months ago they needed to do some simple algebra at work, and a coworker said, "dang, I wish I was good at math."
Somewhere out there Terence Tao is saying he's alright at math but he has nothing on that Euler fellow.