Live data from Hacker News

Data consistency is overrated

two-wrongs.com

71–72 of 72 posts

Re: Data consistency is overrated

#71
post #68

Earlier quoted context omitted.

distinct vs approximate user facing result seems like product question.

The commenter seemed to indicate that even the exact result wasn’t very accurate. Perhaps a failure on their part to communicate this nuance well, or understand the business context.

No.

Everyone involved had roughly a decade of experience with the data in question, the processes that touched it before it reached us and within the company, the various problems with the data and how they impacted the analytics. They also understood how probabilistic algorithms would impact the results.

Additionally we had a team whose main responsibility was to continuously monitor and improve the quality of the data we were receiving. My team had frequent conversations with that team, because anything they couldn't detect would show up in our results.

They just had a fixation on "accuracy" stemming mostly from "our clients wouldn't understand it". Which is interesting because we had frequent complaints from clients who had secondary sources and found differences between what we provided and what their other sources provided.

Re: Data consistency is overrated

#72
Oh yes, because who needs consistency in their data anyway? Why bother making sure that the information we're basing our decisions on is accurate and reliable?

Let's just mix up some numbers and hope for the best.

I mean, what's the worst that could happen? It's not like incorrect data could lead to costly mistakes or missed opportunities. Let's just throw caution to the wind and see where inconsistency takes us. Who needs consistency when you can have chaos, right?

Post reply on HN