Firefox got faster for real users in 2023

* Found this: https://github.com/K3V1991/Disable-Firefox-Telemetry-and-Dat... I haven't compared their list to the one I've used before but it's along the same lines and explains the discrepancy between the config settings and Firefox's actual behavior.

It looks to my naive eyes like the trajectories here are consistent and that the “signal” looks bigger than the variance. The measurements seen consistent with a bunch of smart engineers working to optimize for them during the time period they were measured.

At the same time you’re right: that big August improvement might well have something to do with Northern-hemisphere people switching to lighter vacation reading rather than just the Firefox 116 release happening to drop August 1. I’d share your interest in the longer timescale.

But I have some itching concerns about that methodology with heavy anonymisation and stuff there:

Couldn't it be that the web itself got faster, and folks have better hardware?

I mean, yes, generally these sloppy script kiddies with bloated frameworks produce less and less effective code … but I can imagine, that when some heavily visited page deploys a new better optimized and leaner version (I assume such things happen in reality, don't they?), there is no way to tell it apart from telemetry data telling that everything got slightly faster on average. Or with people getting better hardware, or with OS and driver updates, etc.

There's no reason to assume that what is brought up in a blog post is going to match what engineers are looking at. In this case, I'm a developer at Mozilla, and I would say that I agree that it's worthwhile to look at the 99th percentile as well. And the median (50th percentile). And other platforms. And segregate it by website, but we don't collect that, or by country, but although I think we might be collecting that we avoid correlating too many things before discarding the non-aggregate data. There are too many German users to worry about identifying one by knowing they're German, but there's aren't that many German users with >100 tabs running on Windows 7 on slower hardware, etc.

I wouldn't want to pile up any more data than necessary in a blog post, though. The point would get buried. Man Bites Dog Whose Litter Mate Once Skipped A Veterinary Visit Because Owner Was Out Of Town At A Wedding Between Two People Whose Names Start With D

(Yes, we do consider many different percentiles when making decisions. We kind of have to come up with arguments for what matters, given a change we made or are contemplating. Some things improve the 50th and regress the 95th, for example, and that's a useful clue. Telemetry tracks half a dozen different percentiles.)