Search engines and SEO spam

I'd say it can be useful, but that's not often the case (especially not to the tune of 1000+ words).

Ranking search results on popularity is flawed. It may improve search engine performance and the effectiveness of online advertising but it penalises users who can think critically and independently. There is an underserved market that has been left behind by Google and its "competitors".

PageRank seemed to borrow from the concept of citation count. The idea that "importance" could be measured by the number of times a webpage, like a paper published in a peer-reviewed academic journal, was referenced by other webpages, like other papers published in peer-reviewed academic journals. The initial name for the project before "Google" was "Backrub", referring to the reliance on "backlinks" to quantify importance.

An index of a commercially-oriented www full of sites supported by online advertising is nothing like Web of Science or some other database collection that allows ranking by citation count. The www has no peer-review and no limits on commercial activity.

Google succeeded in creating something highly profitable and sometimes useful, but the founders never delivered on their original promise. That was a search engine in the academic realm, where the technical details were public, and one that would be free from the influence of advertising.^1 Instead the project was turned into an online advertising business. A 180-degree pivot.

The moral/ethical debate went from the question of being advertising-supported to the question of invading the personal privacy of users, for the benefit of advertising. Whatever ideas the founders held in 1998 regarding the influence of advertising on web search were overtaken by the lure of pure financial success. Once oppposed to idea of using cookies for advertising purposes, the founders were persuaded to purchase DoubleClick, a company with a terrible privacy record that uses cookies and purchasing data to profile users as ad targets,^2 for almost double what they paid for YouTube. Not sure what if any moral/ethical debate remains today. While the company is being sued simultaneously by hundreds of plaintiffs, including the US government, one of the founders is "hiding out" on a small island in the South Pacific. Whatever motivations he had to make an open, academic search engine free from the influence of advertising, they seem to be gone.

In sum, the world still needs a decent web search engine free from the influence of online advertising.

1. https://infolab.stanford.edu/~backrub/google.html

Excerpts:

"Up until now most search engine development has gone on at companies with little publication of technical details. This causes search engine technology to remain largely a black art and to be advertising oriented (see Appendix A). With Google, we have a strong goal to push more development and understanding into the academic realm.

Appendix A: Advertising and Mixed Motives

Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users.

For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.

Furthermore, advertising income often provides an incentive to provide poor quality search results.

[T]here will always be money from advertisers who want a customer to switch products, or have something that is genuinely new. But we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm."

2. https://www.nytimes.com/2000/02/17/technology/us-investigati...

https://slate.com/technology/2005/11/why-web-surfers-love-to...

To address your issue, we can simply accept that certain information cannot ever be available day and date. The best results I've ever got from Google is by doing

> site:reddit.com

If I want to know about headphones, or TVs, I'll find better answers in the sidebar than anywhere else on the Internet. But, I will not be able to find those quality answers until a reasonable period of time has passed where products can be tried and reviewed by real people.

The issue is the immediacy. We want answers now, but we won't ever have them until later. This requires a cultural shift in consumerism that companies will not like: "Wait and see."

The same problem happens for people pre-ordering video games that end up releasing in a state of complete trash. You can only get reliable information once the early adopters have tested it. If you are an early adopter, you are shit out of luck, but you are doing the rest of us a service that we appreciate greatly.

I don't think this problem can be solved by another search engine as they currently exist. The problem must be solved by a new kind of search engine that exclusively searches Internet communities (HN, Reddit, etc.). The content must be community produced, since all other forms of writing have monetary incentive. You will find the best results in communities of enthusiasts with respected moderation teams.

So, a curated strategy where the users can UP/DOWN sources they trust for particular topics. Relevancy and user ranking of answers determines score. Of course, the user can search any sources they like, but a voting system would control the defaults.

I think this avoids slanted results as well, because topics are objective. The subjectivity will be in the comments, where they belong. That's in contrast to today where SEO scams can determine how high up results are.

So, to game my proposed search engine, you need to infiltrate the users. I believe this is harder to do across numerous sources compared to the current system, which is game the algorithm.

Ranking search results on popularity is flawed. It may improve search engine performance and the effectiveness of online advertising but it penalises users who can think critically and independently. There is an underserved market that has been left behind by Google and its "competitors".

PageRank seemed to borrow from the concept of citation count. The idea that "importance" could be measured by the number of times a webpage, like a paper published in a peer-reviewed academic journal, was referenced by other webpages, like other papers published in peer-reviewed academic journals. The initial name for the project before "Google" was "Backrub", referring to the reliance on "backlinks" to quantify importance.

An index of a commercially-oriented www full of sites supported by online advertising is nothing like Web of Science or some other database collection that allows ranking by citation count. The www has no peer-review and no limits on commercial activity.

Google succeeded in creating something highly profitable and sometimes useful, but the founders never delivered on their original promise. That was a search engine in the academic realm, where the technical details were public, and one that would be free from the influence of advertising.^1 Instead the project was turned into an online advertising business. A 180-degree pivot.

The moral/ethical debate went from the question of being advertising-supported to the question of invading the personal privacy of users, for the benefit of advertising. Whatever ideals the founders held in 1998 were overtaken by the lure of pure financial success. Once oppposed to idea of using cookies for advertising purposes, the founders were persuaded to purchase DoubleClick, ground zero for the explosion of online ads, for $3.1 bilion. Not sure what if any moral/ethical debate remains today. While the company is being sued simultaneously by hundreds of plaintiffs, including the US government, one of the founders is "hiding out" on a small island in the South Pacific. Whatever motivations he had to make an open, academic search engine free from the influence of advertising, they seem to be gone.

In sum, the world still needs a decent web search engine free from the influence of online advertising.

1. https://infolab.stanford.edu/~backrub/google.html

Excerpts:

"Up until now most search engine development has gone on at companies with little publication of technical details. This causes search engine technology to remain largely a black art and to be advertising oriented (see Appendix A). With Google, we have a strong goal to push more development and understanding into the academic realm.

Appendix A: Advertising and Mixed Motives

Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users.

For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.

Furthermore, advertising income often provides an incentive to provide poor quality search results.

[T]here will always be money from advertisers who want a customer to switch products, or have something that is genuinely new. But we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm."