Do data-driven companies win?
benn.substack.com
Do data-driven companies win?
1–10 of 63 posts
Re: Do data-driven companies win?
#2Define "win".
Is it about making money? Or is it about about using your money to gain greater control? Or some combination of both?
This article assumes its all about the money. But if - as I believe - we are under some hybrid corporate-governance system, where these 2 domains are working together whilst appearing to be separate, then the measure is not just money.
Re: Do data-driven companies win?
#3I worked at a self-described "data-driven" company, and the analogy senior leadership liked to make was that the company was like a machine learning algorithm, using data (particularly A/B tests) to do "gradient descent" of the product into its optimal form.
My first take-away was that using data to make decisions is tremendously, tremendously powerful. A/B tests, in particular, can help determine causality and drive any key metric you want in the direction you want to. Short-term, it seems to work great.
Long-term, it fails. Being purely data-driven without good intuition and long-term bets (that can't be "proven" with data), and the product loses its soul. You can (and should) invest in metrics that are more indicative of the long-term. And you should use data to help guide and improve your intuition.
But data is not a substitute for good judgment, or for a deep understanding of your users and their problems, or of "where the puck is going". It's just a tool. It's a very powerful tool, but if it's your main or only tool, you will lose.
Re: Do data-driven companies win?
#4I see too many "data driven companies" that are not data driven, but selectivly use data to do what the executive wants to be done.
On top of that I see companies with employees who didn't understand college mathematics but now want to be data driven - E.g. basing major decisions on 10 customer feedback data points. Or who don't know the difference between median and average.
Re: Do data-driven companies win?
#5I see too many "data driven companies" that are not data driven, but selectivly use data to do what the executive wants to be done. On top of that I see companies with employees who didn't understand college mathematics but now want to be data driven - E.g. basing major decisions on 10 customer feedback data points. Or who don't know the difference between median and average.
There's also something to be said about having the wisdom to understand which data you can do something about and which you can't.
E.g. if the metrics of customer acquisition vs customer LTV don't make sense on a fundamental level, it doesn't matter what your other metrics are showing.
Re: Do data-driven companies win?
#6I think "dashboard" is perhaps a better analogy than some even realize. When I hear someone describe their startup as purely data-driven, I picture someone driving a car while focusing exclusively on the dashboard.
Data is valuable. Some KPI's can show important, even vital information -- often not directly but through the first or second derivative. If something changes rapidly, you probably want to take notice.
Overall though, I think the metrics that we have available are about as helpful to a company as the metrics we can track about our bodies are to our health. Sure, monitor your weight, HRV or even your blood sugar levels and get some insight. Let it help inform your decisions. But only that.
Re: Do data-driven companies win?
#7We use the term data-driven to describe certain parts of our product wherein we use SQL queries against faithful models of the domain to make deterministic, traceable choices.
Certainly works for us, but I don’t think this is how most people understand “data-driven” anymore.
Re: Do data-driven companies win?
#8I connect data-driven to A/B testing. And A/B testing for me is like squeezing the maximum out of an orange, instead of making more oranges.
It will give a short term boost, but eventually you will have to look beyond your data and explore new things your customers really want.
Re: Do data-driven companies win?
#9I connect data-driven to A/B testing. And A/B testing for me is like squeezing the maximum out of an orange, instead of making more oranges. It will give a short term boost, but eventually you will have to look beyond your data and explore new things your customers really want.
A/B testing can definitely make product teams feel like they're making progress but it should not be a team's only strategy.
It's a bit like moving the furniture around the same room.
Re: Do data-driven companies win?
#10I believe everything else being equal data-driven is better.
Of course if data-driven means short-term with bad KPIs this version of data-driven will fail. I think a lot of people picture this A/B short-term testing when they think of it.
A data-driven approach is still far from objective and relies on choosing good metrics to optimize for.
To this end I would recommend the book "How to measure anything" from Douglas W. Hubbard (https://www.goodreads.com/en/book/show/444653.How_to_Measure...)