This result isn't even peer-reviewed yet. All of this hype about the X17 particle is coming from a pre-print on the arXiv.
https://arxiv.org/abs/1910.10459
They claim a 7.2 sigma result, but the distribution of the null-hypothesis (called PDF(exp) in the paper) was determined from simulations of one atomic transition and "external pair creation on the target backing", and the only source of systematic error they took into account was the position of the target relative to the beam-line. It seems a little paltry to me in terms of trying to take into account possible sources of error... And given that the excess appears in only a particular angular region, they seem to be ignoring the Look-elsewhere effect ("oh, no excess signal here? Let me look elsewhere...") and over-stating their significance. The paper also is riddled with spelling and grammar mistakes, and while that doesn't have anything to do with the science, it does say something about the sloppiness of the researcher, I think.
Anyway, I can't imagine this passing the muster of peer-review in its current state. But I'll be curious to see the revised paper with more details included (if they can actually get it published).
EDIT: They did explain how they determined the background distribution, I edited my comment above.