I mean, when you think about it, it's kind of obvious that an entity extracting revenue from selling advertisement impressions is not going to produce a very useful search engine — it's going to produce a search engine that maximises advertisement impressions. Either by encouraging the advertisers to game the search engine, or by simply preferring the paying advertisers' results, or both, on some level.
So you end up with a Google that prioritises 2-3 promoted results above actual search results (some of which represent the opposite of what you're looking for!) and everything beneath is either a massive mainstream content factory, a Reddit/SO/Quora thread, or any one of a billion terrible blogs/news hosts that contain no content or simply regurgitate someone else's content with adverts, modals, etc galore.
In fact, the only reason why Google's search engine is fairly safe in its product space — the considerable head start on potential disruptors notwithstanding — is that there apparently exists no comparably successful method to extract revenue from running a search engine.
Not that many people are realistically going to pay a monthly fee to have a Google without the noise, despite the amount of noise there is (I probably would). And let's face it, if there was a market for that, Google Premium would probably exist already. Talk about vertical integration if they did though. Help pollute the ocean of the internet then sell people premium membership to sail across it rather than swim in it.
One of the biggest things that Google did wrong was try to act as curator. The moment they start screening results by compliance with Google standards, introducing stuff like AMPHTML etc, anything like that is the moment they make themselves no different to Facebook, Twitter and every other walled garden community.
The internet is supposed to be about broadcasting information, about exploration and chasing the horizon — not locking information behind forced memberships of social networks and paywalls, tardis-like megastructures where you're encouraged to become locked in yourself.
All that should matter is that a webpage's content matches a query. Not whether their website matches some random person's idea of good UX etc. Does the content match the query. That's it. Refined obviously to assess whether a page's content is too insanely well matched (old SEO bullshit) and maybe include that domain rank stuff (if it was based on conversions so it's self-moderating rather than Google saying "[majorpublication].com is better than [randomblog].com because big business website > random person's website")
Google could have been worse, of course. AMPHTML is pretty much over, right? They track your data, yes — but show me the company that doesn't do that. Apple? Apple probably do, they just track less or whatever. Just because it's in the marketing doesn't necessarily mean that they don't do it, it just means that they know people will buy their shit if they say they don't, or make a point of doing it less.
I don't blame Google for the way their search engine has turned out. As others have said, a big part of it is the sheer amount of noise out there now. But more importantly, the way capitalism works causes most companies to produce increasingly shitty products over time. This eventually creates the opportunity for someone new to release a great product, and eventually their great product becomes a good product, and eventually that will become a shitty dividend-paying product, and the cycle will continue.
(to clarify, the opportunity isn't just there for competitors, the opportunity also exists for Google to sort their shit out too).