Personally, just from the implausible level of first-person detail throughout I thought it's immediately clear the book is exceptionally well-researched but still technically qualifies as historical-fiction. We don't necessarily know what these people thought/ate/did minute by minute on the important days at that level.
But I think the biggest "sin" in terms of mixing fact/fiction was mostly implied and not actually stated. What's implied is that Groth saw inside mathematics some kind of terrible truth that motivated him to stop working and withdraw from the world. I don't think it's stated explicitly, but due to proximity with other topics in the book, reader is invited to conclude that there was a discovery of some kind inevitable doom, possibly a super weapon, etc.
We don't know that, but in a lot of ways it might be more surprising if he never thought along those lines. My understanding is that the other limited sources really do say he was talking to God in dreams, preoccupied with apocalyptic visions, became more interested in physics, politics, religion, the problem of evil, hostile entities ambiguously demonic, etc