Here’s what I’m thinking: The neural network doesn’t have to be correct about which one you pick, it has to be correct about which you don’t pick. Only one option they pick is a loss, so if the other party can be somewhat certain you won’t pick a specific option, it can at least tie. So if a randomizer has pretty even distribution, I think it can win more than half the time, because it can gather roughly how likely the same choice is to be played in a row.
I’m basically suggesting a predictable distribution can be exploited in RPS.
Honestly, the bot could always be winning against the RNG by dumb luck. More experimentation would be needed to be sure. I am just making guesses.