Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do data-driven companies win?

benn.substack.com

Post reply on HN
Print view View original

51–60 of 63 posts

Re: Do data-driven companies win?

#51

Post by mrweasel »

itsoktocry wrote: post #39
mrweasel wrote: post #29

Earlier quoted context omitted.

> I believe everything else being equal data-driven is better. While I understand your point, it also depends on what you mean by "better". It's certainly safer, less risk, more predictable and makes planning easier. It also limits creativity and high pay offs. Some one else commented that it was like "driving by looking only at the dashboard". You can do that very safely in planes, but you're limited in where you ca…

> It also limits creativity and high pay offs. Why would being data-driven limit either of these things?

Because being 100% data driven means that you're removing ideas and opportunities that exist outside your datasets.

It's not realistic to have a dataset large enough to cover all situations, especially not those areas and opportunities that you don't know exist. That's also way Googles 20% projects was such a good idea. It allowed to business to grow and develop in directions they couldn't possible predict.

Re: Do data-driven companies win?

#52

Post by thenerdhead »

I think it's fairly safe to say that most true data-driven companies sustain their innovation and business over the years. They have little incentive to take many risks and will hold their own throughout time.

That's to say that real innovation is being more data-inspired. Taking risks with what is known based on your intuition/opinion.

A timeless read on this topic is Clayton Christensen's "Innovators Dilemma".

I think the exact same phenomena applies to investing too. You can sustain your growth by making moves purely off data, but your big wins may come from being "data-inspired"

This blog somewhat covers these ideas in more detail: https://amplitude.com/blog/data-driven-data-informed-data-in...

The irony in this all is that nobody can tell the future and therefore what we know in the current moment is the best thing we've got. Even those with more experience, resources, and knowledge can be disrupted. That's the beauty of the world.

Re: Do data-driven companies win?

#53

Post by itsdrewmiller »

kristiandupont wrote: post #6

I think "dashboard" is perhaps a better analogy than some even realize. When I hear someone describe their startup as purely data-driven, I picture someone driving a car while focusing exclusively on the dashboard. Data is valuable. Some KPI's can show important, even vital information -- often not directly but through the first or second derivative. If something changes rapidly, you probably want to take notice. Ove…

https://en.wikipedia.org/wiki/Instrument_flight_rules - driving entirely by instruments can be important too!

Re: Do data-driven companies win?

#54

Post by corrral »

itsoktocry wrote: post #36
bratbag wrote: post #20

Earlier quoted context omitted.

Agreed. Data is useful for proving my decisions once I have made them, so what's most useful is having short feedback loops so can I catch bad decisions quickly, or reinforce good decisions.

> Data is useful for proving my decisions once I have made them I can't figure out what this means, but I have a feeling it's the exact opposite way you should be incorporating data into your decision making.

I took it as:

Prove (?), v. t.

1. To try or to ascertain by an experiment, or by a test or standard; to test; as, to prove the strength of gunpowder or of ordnance; to prove the contents of a vessel by a standard measure.

Thou hast proved mine heart. Ps. xvii. 3.

(Webster's 1913)

Which makes sense to me. Try something you think might work, then test it—is just plain old experimental science.

Re: Do data-driven companies win?

#55

Post by marcosdumay »

Ozzie_osman wrote: post #3

I worked at a self-described "data-driven" company, and the analogy senior leadership liked to make was that the company was like a machine learning algorithm, using data (particularly A/B tests) to do "gradient descent" of the product into its optimal form. My first take-away was that using data to make decisions is tremendously, tremendously powerful. A/B tests, in particular, can help determine causality and drive…

"Intuition" and "judgement" isn't really the best way to think about this.

The entire problem here is that when people say "data-driven", what they actually mean is "model blind". If you make decisions based on a model, nobody will say it's data-driven, it doesn't matter how empirical the model is. Yet, if you rephrase the question as "Do model blind companies win?" the answer is obvious.

Running incoherent experiments testing for each little event you may think of is certainly better than walking at random. But as everybody knows for centuries, you use data to improve your model, and use your model to improve your craft. Jumping over the model part is an incredibly lousy use of data.

Re: Do data-driven companies win?

#56

Post by jimmont »

marcosdumay wrote: post #55
Ozzie_osman wrote: post #3

I worked at a self-described "data-driven" company, and the analogy senior leadership liked to make was that the company was like a machine learning algorithm, using data (particularly A/B tests) to do "gradient descent" of the product into its optimal form. My first take-away was that using data to make decisions is tremendously, tremendously powerful. A/B tests, in particular, can help determine causality and drive…

"Intuition" and "judgement" isn't really the best way to think about this. The entire problem here is that when people say "data-driven", what they actually mean is "model blind". If you make decisions based on a model, nobody will say it's data-driven, it doesn't matter how empirical the model is. Yet, if you rephrase the question as "Do model blind companies win?" the answer is obvious. Running incoherent experimen…

Intuition and judgement come from changing the context of the analysis--the time range/history, the details included. Geopolitical and economic forecasting appear to fit this pattern. The question posed about whether data-driven companies win seems simplistic and formulaic to me personally. Often outcomes can be better fortunes and circumstantial than from over analyzing and using the right approach. Of course both together tend to be where we'd see success looking backward.

Re: Do data-driven companies win?

#58

Post by mountainriver »

KingOfCoders wrote: post #4

I see too many "data driven companies" that are not data driven, but selectivly use data to do what the executive wants to be done. On top of that I see companies with employees who didn't understand college mathematics but now want to be data driven - E.g. basing major decisions on 10 customer feedback data points. Or who don't know the difference between median and average.

Yup just ran into this at my company. All of the higher ups claim they want the org to be data driven, I then presented them with exhaustive research on a potential tool to use, but they pushed it aside because it didn’t confirm their biases.

Bad execs will find a way to manipulate any good philosophy toward their control

Re: Do data-driven companies win?

#59

Post by webnrrd2k »

marginalia_nu wrote: post #40
deltathreetwo wrote: post #32

Earlier quoted context omitted.

The churn is the result of the fact that we don't understand what is optimal. We know how to optimize for things like the shortest distance between two points but we don't understand how to optimize Software Design or even GUI. This is literally what "Design" means. If something needs to be designed it means there's no theory or notion behind it on what it means to be "optimal." So we "design" a "better" solution but…

Honestly, this comes off more as a model problem than a data problem. Here are some dubious assumptions: * There is an optimal answer. * Human preferences are transitive (if I prefer A over B and B over C, then I prefer A over C). * Human preferences are internally consistent. * Human preferences are stable over time. Apply these assumptions to finding an answer to the question of what you should make for dinner, and…

> Apply these assumptions to finding an answer to the

> question of what you should make for dinner, and you'll

> quickly see there are problems.

And, to amplify your point a bit, there are problems with A/B testing for which of my children I should love the most, or what I find beautiful or interesting; or should I A/B test out whether enslaving others works out well for me?

I chose extreme examples, but the point is that there are many, many things in human experience that don't lend themselves to easy, simplistic rules. It's genuinly hard to work through a lot of a lot of the issues that people face in real life.

With that said, it's it's also important to work through and to understand objective data as best as possible. For clearly defined and nicely-behaving problems objective data is certainly the way to go. The problem is that a lot of the problems people actually face aren't so easy to understand, and aren't so well-behaved.

Re: Do data-driven companies win?

#60

Post by yobbo »

Ozzie_osman wrote: post #3

I worked at a self-described "data-driven" company, and the analogy senior leadership liked to make was that the company was like a machine learning algorithm, using data (particularly A/B tests) to do "gradient descent" of the product into its optimal form. My first take-away was that using data to make decisions is tremendously, tremendously powerful. A/B tests, in particular, can help determine causality and drive…

With A/B tests, the only decisions you can make are among alternatives that are very close to the base alternative.

Generally, you can only generate data and do experiments based on your present state.

Post reply on HN