Data will not tell you what to do
mikkeldengsoe.substack.com
Data will not tell you what to do
1–10 of 75 posts
Re: Data will not tell you what to do
#2[deleted]
Re: Data will not tell you what to do
#3Good insights. Before the term was bastardized, then consumed, by Machine Learning, the experimentation component was considered the killer delivery by data scientists (putting the _science_ in the term). Now, most folks assume MLE := DS, rather than MLE ⊂ DS
Re: Data will not tell you what to do
#4As painful as it is to admit, I've come to the same conclusion, after wasting quite a lot of time and efforts.
As Warren Buffett likes to say - "It's better to be approximately right, than to be precisely wrong"
This should be a poster in every company doing any kind of data.
Re: Data will not tell you what to do
#5> Intuition is underrated
> Spend time where your customers are and make your own conclusions.
This is a great article, very well-written, and I enjoyed reading it. However, could intuition and spending time with customers be considered another way of collecting data points to inform data-driven decisions?"
Re: Data will not tell you what to do
#6> Intuition is underrated > Spend time where your customers are and make your own conclusions. This is a great article, very well-written, and I enjoyed reading it. However, could intuition and spending time with customers be considered another way of collecting data points to inform data-driven decisions?"
Generally when talking about data driven you are talking about quantitative data and not qualitative data.
Re: Data will not tell you what to do
#7I've never worked at a company where we were anywhere close to being able to A/B testing on customers. We just always shot from the hip. I think if you are doing that kind of testing may e you've run out of helpful ideas?
Talking to real customers and helping them solve real problems is really potent. And you can get more than just the color of a button. You can get the direction your company needs to go for months.
I think part of the problem is that science takes too long. It's like waiting for evolution to play out. You're company is at war with everything, entropy, the economy, your competition, the attention span of customers. Do you have time to science your way to success? Probably not. Do you have time to gamble on your intuition? Barely.
Collecting data isn't bad per se. But you should always be asking yourself if you are solving the right problems before you waste your time on it.
Re: Data will not tell you what to do
#8I've never worked at a company where we were anywhere close to being able to A/B testing on customers. We just always shot from the hip. I think if you are doing that kind of testing may e you've run out of helpful ideas? Talking to real customers and helping them solve real problems is really potent. And you can get more than just the color of a button. You can get the direction your company needs to go for months.…
I was at a small games company that was way too small (in audience and headcount) to be A/B testing. They landed at a local maxima that was nowhere close to a sustainable business. The leadership was way too afraid to make actual decisions to actual problems and opted to change the color of the button for a few years until they ran out of money. ‘Twas a tragic waste of a team and money
Re: Data will not tell you what to do
#9Maybe I’m missing something. The goal of an A/B test is to test a hypothesis. Where that hypothesis comes from is irrelevant. Sure you can waste your time testing stupid hypotheses that don’t have a lot of business impact, but that’s beside the point.
Re: Data will not tell you what to do
#10Data may not tell what to do next, but models do. For example Bayesian optimisation framework tries to do exactly that.