I do think that the statement 'Men are on average more prone to violence than women' can be sexist, depending on context.
However, I think that it is possible to contextualize fact-based-generalizations to remove sexist/racism from them. If you say 'society conditions men to be less emotionally open, thus men end up being more violent, see these psych studies', that's all well and good. On a similar note, if you say 'Jamaicans on average grow up in higher poverty with worse education, and more end up involved in crime', that contextualizes it to show that you are not speaking about the nature of the people, but rather about specific situations and facts.
A final factor you're ignoring is that there's always a much broader social context through which your generalization is interpreted. "Black fathers are lazy" is a more racist generalization than "white fathers are lazy" because the first has racist context associated with it, and the second has much less of such a context. This context may be global or regional, though with the dominance of american media, that context is basically global at this point.
On a similar note, anything to do with a specific people or race needs more explicit context because there's so much racist ambient context.
> Seems like this argument is more about each side shutting down ideas they are uncomfortable with.
I am not okay with anyone stopping you from providing facts and ideas about crime in different societies, just trying to stop you from generalizing in such a way that implies one race is inferior to another. This isn't a matter of discomfort per-se, but rather a matter of us having already seen the end of the slippery slopes such talk, when weaponized as rhetoric and internalized, leads to.