The world needs a non-profit search engine

Beyond that, complaining Google does not do XYZ misses the point. Google is a search engine designed for the average user and the average user does not want verbatim search. They also do not want: advanced search operators, true Boolean search, regular expressions, API access to search, open source code, real-time streams of pages Google’s crawling, etc.

What they do want and always have is natural language based searches in there language of preference with clarifying responses from the search engine in natural language; that is, they want to treat a search engine like a person and be treated like a person; which was odd that they referenced Knowsmore, since Knowsmore [1] used keyword based searches, not plain language searches.

Google is not the primary problem, the average user is the issue. Unless people realize that — they’re fighting in a war they do not even understand.

To make it even more clear, Google is easily able to detect and block users blocking ADs, but they do not. More than 60% of users still don’t block ADs; not because they love ADs, but because effort to figure it out simply is not worth it to them, they like ADs, etc.

[1] https://m.youtube.com/watch?v=T3wiGSXbeQE

Not so in Bing, it's simply not there. The page exists since March 2021. Bing Webmaster Tools reads "Discovered but not crawled. URL cannot appear on Bing", giving no further reason. Also: "Last crawl attempted 01 Feb 2022 at 19:35", which means that Bing did not bother to retry for months, despite me submitting it manually on a regular basis. Clicking the "Live URL" tab results in entirely green checkmarks along with "URL can be indexed by Bing".

Another example would be my personal site: https://herrbischoff.com. Same issue. That one is listed on Google for more than 10 years.

Linking an archived copy of the content has merits, though increasing The Usual Suspect (the Internet Archive's Wayback Machine) itself has difficulty in preserving complex content. (The rather less transparent Archive.Today is often superior in quality if not necessarily in trust or reputation, and I say that relying on it heavily myself.)

I know that I've noted news organisations which do seem to link sources relatively freely, and thought I'd commented on it previously. I'm not finding any previous mentions by me here at HN (my usual personal quips trove, amongst other benefits). Though if memory serves, the New York Times tends not to provide links where it really ought to. I believe the Los Angeles Times might have a greater tendency to. Perhaps also NPR and/or The Guardian which I tend to rely on, though I'm not certain that's been my earlier observation.

In the 19th century and through a good part of the early 20th, newspapers would not only reference specific documents or speeches but publish them in whole. In large part, this is because that was the only way to distribute and reference the material. That practice has waned tremendously, and we often see material only through commentary and reference, rather than in original form. I've come to view this quite dimly.

I'm also fairly certain that copyright is a major consideration and factor here, and another way in which it's proving a disservice to the public good.

Let's try? It's also an interesting research topic in itself and might be a topic for academic research. At the moment Google is a black box and their incentives are not really aligned to stop SEO. It's good for them to show more ads, it's good for them to show you copycat pages of github/stackoverflow with ads. Not saying that Google is doing this on purpose - I doubt it - but we don't know. It's surely possible to create an index and ranking that prefers different things than Google.

Let's try something at least. It will be gamed, will be worse probably but it's open and can be a playground for academic research.

Best that can happen is that there are ways to for a better ranking and Google was dishonest to maximize profit. If it's still gamed at least the mechanics can be studied and analysed and maybe someone can figure out a switch like 'be unfair in ways Google can't' - would love a 'no ads on the page' switch that would probably solve quite a few problems.

Trust our black-box and you won't have enough devs for this is just a bad answer for such an important problem. The amount of stupid simple redirection spam in my results in the last few years also looks like that Google just doesn't care alot about this anymore.