Common philosophical answer is that people only can know the approximation of reality, never the reality itself.
Better approximations usually score you more points, which is why our perception usually evolves closer to reality.
Bad scoring systems exist.
For example, let's say if an extremely privileged person – a favorite son|daughter of a billionaire – takes random chances and people around him|her always agree and accommodate for them to win (because they want a favor from a parent), than this person will most likely have a very warped sense of reality through no fault of their own. Basically their only approximation could only be "Do whatever, just don't touch fire with bare hands and don't make daddy angry"
On the other hand, the suffering child from "The Ones Who Walk Away from Omelas" would also have warped sense of reality, because they lose no matter what action they take. The only reality approximation they can know is "everything feels bad"
Sidenote: in my philosophy, the continuous warping of reality happens mostly to people in overprivileged and underprivileged positions. Which is why I believe that a good equality index makes for a healthier society. But thats my pov, I'm not trying to sell it or anything :)
Note that I wrote "continuous warping". While sometimes bad approximations got rewarded more than good ones, reality is probabilistic. Since it can't be comprehended by people, we basically define reality as "something that is more likely".
And that's the answer to your question. If you tune past the reality, you start scoring worse again, and turn back. It's a distance thing, and the goal is to get as close to zero as you can, doesn't matter from which direction.