That quote has always bothered me. He uses incredibly small details to deduce incredible things, but thinks the solar system will never have any influence upon his casework. I can imagine scenarios where people thought the wrong things because of what the sun was doing, how far the sun was from the earth at the time, etc etc. It
does matter, just not often... Much like the rest of what he uses to deduce things.
It's like saying he doesn't care what clerks do in their jobs, but then deducing that someone's a clerk by their hands. He's have to know what they do for that, even if it doesn't otherwise affect him.
That said, I agree with you... Mostly. I don't really care what someone believes, but there are some beliefs that tell me people aren't critical thinkers and will just believe things they're told... And then stick to it no matter what. I definitely treat those people differently for my own protection.
If flat-earth was the only thing that person believed that was weird, I'd likely just laugh and overlook it. But so far, I've never found that to be the case. They go in for a lot more.