I worked at Facebook for some time, and did a bunch of data work. We had this culture of building something, looking at the results between the experiment groups, and then choosing the statistically more successful one -- i.e. a newsfeed algorithm that had better engagement. This sounds great at first, and certainly is straightforward if you want a promotion. But behind the scenes some of us had this thought that our…
Online Analytics is now really nuanced - you need to know what metrics are important for * your * business, not just use boilerplate kpis.
For instance, an e-commerce website is clearly looking to lead the user to a lot purchase and the more they purchase, it’s good for the business so kpis around sales conversion help and recommenders help your business increase sales.
For Netflix though, the users have already paid for the service after which they land on the website. Most users I imagine then expect to be provided all that Netflix has to offer in an easy way. So if I was a Netflix product owner, I’d be more interested in Kpis around search-ability, having an anti algorithm that “suggests” completely random obscure shows, “switchability” of users - how less of a time do users spend on a movie or show.
I imagine they’re doing this but as a user I don’t see this at least - they show the same old stale recommendations for me, I’m always trying to hack their search to find what I want and they continue to invest in content that’s mostly miss than hit. I wish they at least had a directory for me to browse through (at least I’ll be driving their engagement metrics to help them drive their valuations)