For those missing context, "Gender Identity Disorder" in the DSM-IV, which described transgender identity itself as a mental illness, was replaced with "Gender Dysphoria" in the DSM-V, which was scoped specifically to the depression/pain of feeling socially in the wrong gender role, as well as the feeling of one's body being wrong.
The DSM does cite academic sources IIRC, but they're listed as a while for the book, rather than being referenced by specific sections or criteria, which is a common critique of the DSM in general. Additionally, Ray Blanchard, an academic who believes transgenderism is caused by "autogynephilia"/"autoandrophilia" (i.e. a sexual fetish caused by attraction to oneself as the desired gender), was a member of the committee who decided the DSM-V definition.
So no, OP, academics who didn't support trans politics weren't scared off by the community.. Ray Blanchard played a big role. As for the "minimum of real science," there's a lot of sociological and psychological work that goes into the DSM. Some of the pressure was certainly political/social, like the removal of homosexuality was decades ago, but there are sources listed in the back of the book, even if it takes some digging.
Disclaimer: am trans, am really done with justifying my existence but I felt I should share what I know.