If you read his last two novels he was clearly well-informed about mathematics. However I have to assume that McCarthy didn't actually master all the material in the math books mentioned here, I think the reporter may be a little too credulous about that. I suspect he had the very common experience of buying a yellow book and being defeated in the first couple chapters.
I'm not saying that he did, but this along with being the right age to have read How to Read a Book by Mortimer J. Adler strongly suggest that he used that book to grasp a lot more of his books than most people can.
That book gives you a very good strategy for reading books that are beyond you normally. In the three years since I've read it I've managed to finish books that I couldn't read even when I was doing my PhD and it was my full time job to understand them.
The funny thing is that I only ran into that book when I was trying to figure out how to build knowledge graphs for complex documents using LLMs. Using multiple readings to create a summary of each chunk, then a graph of the connections between the chunks, then a glossary of all the terms and finally a critique of each chunk gave better than sota results for the documents I was working on.