That’s part of it: when you got a new phone, you probably noticed it being faster than the old one but over time your expectations adjust.
One factor which used to be under appreciated was battery degradation prior to that whole “Batterygate” flap back in 2016 where a lot of people learned that iOS throttled processor performance when the battery could no longer supply enough voltage for peak performance. That’s a lot more visible now so it’s less of a surprise than it used to be.
Most of the problem is that developers aren’t focused only on performance, so as the baseline hardware capacity increases the apps will slowly start to use more since very few people are going to spend time on something which seems fast enough.
That doesn’t happen all at once but it adds up over the half decade a phone will last. If you bought a phone and never updated anything, its performance would seem far more consistent … at least if you could avoid getting malware installed.
That third point is why I shared the link above: a lot of developers upgrade more frequently than average people and that means that our instincts for what seems fast enough might be missing things with our apps. That usually doesn’t mean things are unusable but it’s still polite to use your work on, say, the phone a senior citizen gets subsidized to make sure that you’re comfortable with that being the public face of your work.
(Bandwidth usage is at least as important here, too: use your website on 3G or ask how much it’d cost to use on a metered plan)