I think this debate is too abstract to be useful. Certainly, there are some cargo cult practices where various forms of "key rotation" has gotten out of hand for no real benefit. But there are many valid scenarios which depend on different overall system characteristics. I'll just revisit a few.
1. Actual encryption key rotation. This is one grandparent of modern session limits, built with an assumption that keys might be compromised by cryptanalysis rather than stolen by malware-infected clients etc. Rotation reduces the number of ciphertext examples made available to perform the cryptanalysis, thereby reducing the risk of successful attacks. It also may reduce the scope of compromised data if and You may have a UX requirement for preventing idle sessions to stay open on clients because people are in a shared space with client devices, i.e. HIPAA touches on this for medical systems. only if key exchange is still considered secure, such that holding one key does not let you decode traffic captures to determine earlier/later keys in a transitive attack.
2. Forcing idle logouts with less dependence on client systems. This is another grandparent of modern limits which is sometimes cargo-culted. The underlying risk is people logging in and then abandoning a terminal in a shared space, so you want to reduce the chance that someone else can walk up and start using it. You really want the client system to participate in this, i.e. not just end a session but lock/clear the screen so previous data is not on display for unauthorized observers. But it is often seen as defense in depth to also abort the remote session credentials so that future actions are unauthorized even if the client software has failed to do its part, such as if it has crashed/hanged. This one has the weakness you mention that a malicious client could do endless refresh to prevent the detection of an idle UX by the server.
3. Forcing reauthentication periodically or for high-value actions. This is more paranoid than the prior idle logout concept, and actually demands the user reestablish their authenticity during an active session. This has been used historically as another kind of defense in depth attempt to verify user presence with less trust of the client system. But it is also used as a UX ritual to try to get the user to pay attention as well as build an audit chain for their consent to a specific action. Designers might tie this in with MFA/hardware tokens, to get different kinds of user-presence detection throughout a session.
4. Decentralized "web-scale" architecture and caching. In a traditional system, a session key is little more than an identifier and might be checked on each use, i.e. each web request, to determine actual client identity and privileges with a server-side lookup. But as people developed more distributed services, they have often shifted to embedding more of this info into the session "key" itself, as a larger signed blob. Different service components can decode and operate on this blob without having to do synchronous lookups in a centralized session database. This is where automatic refresh still serves a purpose, because it is during each refresh handshake that the centralized check can occur and potentially reject the refresh request or issue modified privileges. This rotation rate defines a system-wide guarantee for how long it takes certain changes to take effect, e.g. disabling a user account or changing the roles/privileges assigned to a user. I have seen this approach even in systems where the session key is a mere ID and cache invalidation could arguably be handled in the backend without forcing client's session keys to rotate. This seems like cargo culting, but is useful if the system operator wants to retain the option to use signed blobs in the future, and so does not want to allow client programmers to assume that client session keys are stable for indefinite time periods.