Another factor here is when people abuse the word "fact" regularly. If you ever get into a discussion with somebody who claims they've proved their point by citing "facts", then actually follow up on their citations and discover that it doesn't actually say what they think it says it creates an impression of the messenger, not the message. Enough messengers saying the same faulty message over and over and you distrus…
In reality, this makes sense if we think of knowledge in a Bayesian sense; we don't have absolute facts, we have little bits of information we gain over time. When a new bit of information comes in, we have to determine our new reality - what is the chance that our current views are wrong versus the chance that the new information is wrong. If we keep getting new information saying our current beliefs are wrong, we might end up changing out view eventually.