All the easy math has been done. People have been working on Mathematics for a long, long time. To "make meaningful and significant contributions to the field" is one of the hardest intellectual tasks a person can do. It is probably impossible for someone at a midpoint in their lives to suddenly develop an interest in math and make a meaningful contribution. Even people who have been immersed in Mathematics their whole lives often cease to make major contributions in middle age.
You do have to be inherently gifted to be a good Mathematician. You even have to be really smart to be a mediocre one. This is hard stuff, the Olympic Marathon of intellectual pursuits. Do you have to be gifted to run in an Olympic Marathon? Someone that was merely determined might be able to qualify for it, but the ones that win are definitely genetic freaks.
You can certainly enjoy math without being a super-genious. If someone were dogged and creative enough and focused on a new enough subfield, they might even be able to contribute a little, provided they've kept their analytical facilities sharp in a field like Physics or Engineering. But then again, anytime a new field opens up there's hundreds of PHD students across the world that jump into it looking for a new bit of math to write their thesis on. It gets fleshed out pretty quick, and the low hanging fruits are the first to go, and the people picking the fruits are PHD math candidates.
Please note that this is just my opinion from being around Mathematicians. I'm not a Mathematician myself, but I was around them when I got an undergraduate degree in Mathematics. That degree taught me that Math was really hard and I would probably have a more significant impact on the world if I pursued something else.