Earlier quoted context omitted.
> Does anyone have any actual statistics or quantitative data on the quality of Google search results? Google has. They use this data expertly to improve search. Common sense and technological advancement tells us that, quantitatively, Google search has become better year over year, for all their relevant metrics/cost functions. And likely, exactly because it has become better for all its users in aggregate, it has t…
This presumes that the metrics they optimize for are intended to represent usefulness to actual users and not, say, ad revenue. Even if they do intend to optimize for usefulness, this doesn't mean that they have metrics that accurately represent that. I also think you're underestimating average users. Anecdotally I've heard my parents complain repeatedly about the incoherent, auto-generated, affiliate link spam that…
They have multiple levers, of which user search quality is a big set. There is always trade-offs and a balance that must be found, which aligns with company vision and strategy. Having these levers allows business-decision makers to direct focus top-down (on a certain set of users, on producing great ad numbers, etc.).
It is clearly hard and important to design these levels and find the right balance, given a rapidly changing company and user-base. So a lot of expertise and power is invested to measure the right things, and to find the right balance (an incorrect/risky balance should also be adjustable with other levers).
So for me: either Google is trying really hard, but essentially failing. Or they have the best of the world, with all the right context, designing these levers. While hard and sometimes wrong, I do not expect to contribute anything which may improve their lever settings. If someone does know, Google would like to hire them.
So while true, that accurately measuring things with proxies, is really hard, and sometimes done wrong at companies. I do not think Google gets this wrong, or at least, gets this to be the best of breed. If their metrics still cause long-term search engine quality loss, would show them to not know what they are doing. I think they do know very well, better than me at least.
I would agree too that the balance of levers right now is in line with Google's strong market position. Search engine quality could take a small hit, if justified with extra adsense income. But when search engine quality noticeably start going down, then all other metrics will suffer. You should have teams with sole focus on improving quality. Other teams will have to realize that favoring their lever over the search-engine-quality lever must lead to worse outcomes for Google in general.
About product searches, I myself was not able to do this satisfactory 15 years back. It improved. But need to stop viewing things as a single lever, a single metric. To say search has become "worse" in general, is to exactly fall into the trap of not accurately measuring and losing too much nuance/details for competing objectives.