Partly, perhaps. For me, a large part is the notation. I know conditionals, I know loops (as in for loops), I know SQL... but if you show me logic, set theory, a sigma, or, uh, what's this sigma but for multiplication, I wouldn't have known what you meant a few years ago. It's not only understanding the math, it's also the notation.
I still don't know how to read things like curly braces or matrices or a dozen other things that I don't even know the name for, and figuring it out is usually more work than it's worth. As a result, I don't understand Wikipedia articles that are supposed to be introductory and understandable to the masses, but because the article's writers are being fancy, it's only understandable for a privileged few. Often the concepts, if I do bother to understand them or look them up elsewhere, are rather simple.
It also doesn't help that people use things incorrectly or ambiguously. Sometimes a line over some math will mean the average (or "mean", to use the ambiguous lingo) and sometimes it's an infinite repetition.
Or the lingo: a friend of my girlfriend was baffled I didn't have X in school and didn't know what it was (I think it's derivations). She kind of explained it to me and I kind of understood, not recognising it. Later that evening I used it, unknowingly, in programming. When showing my girlfriend what I made she pointed it out to me.
Heck, I recently used logarithms fairly intuitively where they were never properly explained to me. I just knew the formula for entropy calculation (log(possibilities)/log(2)) by heart and managed to make something that converts a number from any base into any base (e.g. base 13 to base 19). I can do math, just nobody bothered to tell me about the fancy symbols. To me, the symbols mainly seek to obfuscate, look smart, and ensure job security.