But what if I want a faster horse?

Teams should identify their drivers of key metrics and do power user analysis based on this. A halfway decent analytics team should be thinking this way.

Ultimately, analytics are just a view into the business. This thread is complaining about doctors not using microscopes when diagnosing system issues - sometimes a narrow slice is important, sometimes you need to zoom out. If you focus on your "early adopters" or power users exclusively, without understanding how they affect the business, then you are at risk of building things that most of your user base doesn't want.

Power User Analysis: https://andrewchen.com/power-user-curve/

It's the same with mobile payment. AFAIK there isn't a single bank left in The Netherlands which has its own mobile tap-to-pay app, everyone has switched to Google Wallet.

Good for them that they want to save a few bucks on developers, but why do I have to give my payment info to the devil? It's a third party which has nothing to do with the payment itself, and the fact that some banks used to have their own tap-to-pay apps shows that it clearly isn't a technical requirement.

I'm arguing because I have used /both/ Chrome and Firefox in parallel since the initial release of both pieces of software, including regularly benchmarking them. In the sum totality of the data I have seen, there have been many moments of back and forth where one was "better" than the other, but in the end they are roughly equivalent. When Chrome /first/ released, it had a huge performance advantage explicitly due to V8 and how heavy JS usage was on the web (which has only gotten heavier over time). After that advantage was mostly nullified by the rewrite of the JS engine in Firefox, the performance differential was around a maximum of 5-10% at any given time in one direction or another as both teams worked on improving performance.

> You don't need marketing to tell you that it's a better experience when you could run them side by side and notice that one would crash far more often than the other, and one would struggle with lots of tabs and the other wouldn't, one would render pages faster and more accurately than the other.

As mentioned, I have run them side by side daily for a decade+, including for many long stretches of times both the stable and nightly builds of both. I /still/ to this day, use both browsers every single day. I have not seen anything which would make me believe that one is more stable than the other, or that absent the performance gains on heavy JS sites (early SPAs), that one had a particular advantage in tab-count/memory footprint compared to the other.

Almost all the performance differences were deeply tied to the JS engine, and actually still are (but now wasm too).

> Again, you obviously aren't able to objectively talk about end user experience for some reason and need to be honest with yourself about that. You should load up a VM with XP or Vista and Firefox 3.0 and refamiliarize yourself with the time period you claim to have lived through.

I might do that over the weekend for kicks and grins. I assure you, I am being honest and fairly objective.

It's funny how everyone is so certain I'm wrong, but provided no evidence, other than to point out things that are based /exactly/ on the one major technical win I acknowledged in my original comment and have completely ignored the very public benchmarking efforts that have gone on the entire lifecycle of Chrome.

> So you want them to watch a few exclusive shows a year so they feel like they got their money's worth, while not actually costing netflix much.

No, that's not what the strategy is and they're quite open about it - the strategy is to maximize user consumption for every user, because that keeps them subscribed. I think a lot of people think that they use sophisticated analytics and machine learning etc to decide what to greenlight, but they don't. They use the judgment (and politics, and egos) of Hollywood studio executives (and often the same Hollywood execs that a few years ago were employed in "legacy" media). Although I will grant that they've been innovative in producing/distributing international content, this is really just globalization and labor arbitrage (it is cheaper produce content not in Hollywood, that's not news - they just spend the extra $$$ localizing international content to different global target distribution markets but again, this flow has happened forever, it's just typically been Hollywood -> localization -> foreign market rather than foreign production -> localization -> Anglophone market).

Where analytics and ML does come into play is deciding which things out of their enormous catalogue they push to individual users at any one time - that process is highly reactive, individualized, dynamic - that's why strange and seemingly random media become big hits on Netflix while being largely ignored by the commentariat, and vice versa, why series with dedicated fanbases don't get renewed (the analytics tell you that, despite the apparent success, further investment will not improve user engagement with the platform by enough to be worth the spend).

As I said, it's an extremely difficult problem. To be honest, I doubt it's really solvable in a scalable way across an entire org. The best you can probably do is a combination of implementing a few top down directives and, on the other end, fire fighting flare-ups around specific hot points. But I also hope (desperately) that I'm wrong and that you'll build that shining Camelot on the hill in your org.

Top Down

* Start with clear CEO buy-in supporting a clear manifesto. Include some case study-ish examples of how short-term metric-chasing can go wrong. Do education sessions around this across the product and design orgs. Socialize the concept of "Enshittification." Get people sharing their own examples, whether how Google Search used to be good or how they used to be able to find stuff on Amazon but now the fucking search doesn't even work with quotes or exclusion like it used to. Actually show how you can't find a specifically narrow type of product by excluding features. Ask "How did smart, good people slowly slide down a slippery slope to a pretty evil place?" Discuss how your org can avoid the same fate (or if it even should). Goal: Create awareness. Win (some) hearts and minds.

* Radical idea: seize control of all granular analytics data. Yes, I'm suggesting that product teams cannot directly access their own raw analytics data anymore until it's been corrected for short-term bias and to re-weight by user type. Nor can they unilaterally add new analytics to their product until your CA org has vetted that even gathering that new data won't inappropriately bias internal perception. Before distribution to product teams, granular usage data is first recast and contextualized into new user-type and time horizon buckets that make it hard to chase (or even see) lowest-common denominator "bad" product changes.

I think this is hugely important. I saw certain savvy PMs cleverly manipulate how analytics were tallied and also suggest new measures in a veiled effort to boost short-term incremental metric gains, almost always in the quarter before bonus season. I also saw designers who were heavily bought into the "less density, less choices" zen ethos I called "The Church of Saint Johnny Ive" (which seems to pathologically despise advanced and power users), actively weaponize analytics to generate data supporting their religiously-held worldview and force killing significant functionality beloved by smaller advanced user segments. If those designers ran Burger King the slogan would have to change to "Have it MY way (because I graduated from Stanford D-School and know what you should want)". If you don't seize control of the raw usage data so it can't be weaponized for KPIs (or religious agendas), you'll never be able to make serious traction. Also, doing this will trigger World War III and you'll find out right away if senior leadership is really committed to supporting you. :-)

* Create new segmentation categories of user types. For example, use in-product behavior to identify power users who are passionate and engaged (discount daily frequency and session time / amplify usage depth of specific advanced features), Identify long-time users who were early adopters and dramatically amplify their analytics signal. Every click they make should be worth hundreds of drive-by, newbie users who barely understand the entire product yet.

* Create KPI demerits for teams who make changes that annoy or dismay long-term users as measured by posts on user forums, social media and in deep interviews of unhappy or exiting customers. A handful of such posts should be able to wipe out the gains of a hundred incremental pixel-moving tweaks. Causing strong negative feedback from thoughtful users who care should be feared like touching the third-rail.

* On that topic, once you have control of the granular usage data, simply aggregate all small increases or decreases into one big bucket that's only released into the overall number on a time-delay, maybe even once a year right after KPI/bonus season. Make it so no one thinks they can get "get there" by optimizing 0.1% at a time. All the tweaking of shades of color or moving shit 4 pixels is a distraction at best and at worst ends up losing the beating heart that engages users who really give a shit about the overall experience.

* Assign a tangible economic cost to teams removing a long-time feature. Of course they always have analytics which say "not enough users use it." Institutionalize an organizational default position that's extremely skeptical of removing or moving (aka burying) stuff that's been there since the product's "boost" growth that made it what it is. That shit's grandfathered in and is "don't touch" unless they've got an overwhelming case and a senior product owner ready to make a career-betting stand over it.

* Overall, adjust the KPI/metrics economy through targeted inflation and devaluation of the currency to focus on longer-term objectives.

Bottom Up

* I like your KPI offsets idea.

* Also create a way of rewarding doing more of the right stuff. Special awards not based on specific metrics but on overall "getting it" and making sincere creative efforts to try stuff that's not likely to pay-off near-term.

* Feature user feedback forums more so they get more use. Spiff teams that get more feedback as measured both by quantity and degree of depth. Add specific categories like "Hey, Put That Back!" to encourage that sort of feedback. Don't just count posts and up votes. Inflate the weight of long, passionate or angry posts and posts that elicit more written replies in addition to up votes. Apply appropriate discounts to frequent feedbackers and amplify feedback from people who signed up just to bitch about this one thing. Teams should fear making changes that cause long-time users who rarely post feedback to post emotional rants.

* Find those individuals in the product, design and engineering orgs who believe in valuing the depth of long-term user commitment as much as you do. Make common cause with them. Have a secret club and handshake if you have to but support them and elicit their 'outside-channels' feedback. They're your best source of warning when the forces of short-term darkness are coming in the night with pitchforks (and they will).

Good luck, friend. We're all counting on you!