I had your exact same reaction.
Related story: In my free time I am a coach for a competitive math team, I recently had to design an application to automate multiple choice problem submissions. Nothing stellar, your run-of-the-mill CRUD app. But it was under some interesting engineering constraints: users would have to log in from a variety of devices, phones, tablets, bad chromebooks, etc., possibly on slow cellular connections.
So naturally I went for a simple server-rendered design, with stock Bootstrap hosted on a CDN.
The overwhemling reaction was "this is so fast". Now, being server-rendered with absolutely no XHR, the app spends most of the time doing nothing, but it's interesting laymen think it's fast.
I think we came to associate "modern" designs with piles of javascript, requests everywhere, popups, ads, and the like.
Probably the correlation is indeed there, even if it's obviously not causation.