"For more than a decade now, the fact that people have a hard time gaining actionable insights from their data has been blamed on its size." The real issue is that business people usually ignore what the data says. Wading through data takes a huge amount of thought, which is in short supply. Data Scientists are commonly disregarded by VPs in large corporations, despite the claims about being "data driven". Most corpo…
What “data driven” normally looks like: business people ask questions that are not answerable, but the DS team often goes off and tries to answer them anyway. I’ve been in so many meetings, in multiple organizations, where a senior leader asks a question like, “why did this number go up? We need an owner to deep dive this.” Then, someone disappears for a few hours or days and comes back with some nice narrative that more or less might make some sense, and is directionally consistent. Then, best case, they go “hmm, ok” and worst case, they say it’s an interesting view and we need to add it to the tracking. Then, move on to a new question + action item, rinse and repeat. And then, 6 months or a year later, some brave soul goes, “geeze, there are way too many things to look at here, can we streamline this reporting to focus on the handful of most important metrics, and then assign owners to review the others and surface anything interesting?” And then all those things people asked for over time get stuffed into an appendix or deleted, and the whole thing starts all over again.