Recent evidence tends to indicate that 'not smart enough' is probably a myth. Almost everything can be attributed to exposure and effort at some point rather than some innate smartness. A lot of advanced math takes some serious concentration to understand. For some non-practical aspects, I found that I lacked the motivation rather than ability to understand it . One particular class where I seemed to hit my tolerance…
I have repeatedly gotten tired of math when it seemed like meaningless puzzle-solving. Why do I really care, for example, how many nonabelian groups of order 36 exist? When math seems to me like it's providing a vocabulary for framing and answering deep questions about the world, or making it computationally feasible to find the stamp of causal influence in data or design real things that are impossible to make without out, then the motivation is there.
This might just be an artifact of the way math is often taught. For example, real analysis is often taught as meaningless proof-finding (e.g. proving things that seem either pointless or obvious). But there's a fantastic book, _A Radical Approach to Real Analysis_ by David Bressoud, that teaches the exact same subject matter as the fruit of deep, pressing, and non-obvious questions that stirred debate among mathematicians for around 100 years.