Sure. If you believe that the browser should only be a document reader and not an application platform then go ahead. It's kinda like buying a smartphone and then using it exclusively as a phone though.... I think the real point of irritation here is all the 3rd party junk that gets loaded in that has nothing to do with the main functionality of the site that you are on.
First, while I agree with this being the biggest issue, there is a little bit of nuance to your other point.
> If you believe that the browser should only be a document reader and not an application platform then go ahead
The thing is, the overwhelming majority of those applications are glorified document readers, with a bit of social media plug ins thrown in. It features rich media perhaps, but you don't need JavaScript for an image or video tag. Of the remaining applications, the majority of use-cases is forms to send over the web. This can often be enhanced by JavaScript, but is also not that insane in terms of needs. Then we get to the remaining use-cases of the "application platform".
There's also the part where most websites are just plain badly designed. For example, many sites that have a video that auto-plays when you open it, even if you open a text article and the video is totally unrelated to what you were watching. Worse, some of them even tend to autoplay to a new video, often redirecting your webpage, interrupting the reading experience. Sure, I can pause. It still downloads and buffers a video I never intend to watch. Similarly, whatever happened to progressive enhancement? Websites that give you a blank page unless you download three megabytes of JavaScript. WTH.