Ask HN: Why isn't there a Google competitor emerging?

Google seems to have optimized its search engine with the goal of making it as easy as possible for potential consumers to connect with Google advertising customers. Google also collects data on consumer search patterns again with the goal of connecting the consumer to the advertiser. That's the business model as far as I can tell. If you don't want to be sucked down their engagement hole, you have to clear your cookies after every search and probably going through a VPN is better.

There is an obvious problem here - companies who don't advertise with Google could easily find themselves blocked from appearing on the first pages of Google search results. Would anyone really be surprised to find that the internals of the Google search algorithm have a weight factor that assigns better scores to sites that advertise with Google? Hence requiring Google to expose the internals of the algorithm to public scrutiny really seems like the only way forward. Yes, those are 'trade secrets'. So are the contents of your favorite hot sauce, but government regulators require those contents to be displayed on the product label (for good reasons).

On top of all that, Google's also under pressure from governments and media corporations to push their information content (aka propaganda, influence, etc.) to the top of search results, burying anything like independent content in those areas (world events, domestic politics, etc.) far down. This is particularly obvious on Youtube incidentally, but Google has the same problem. Some of this can be avoided using the 'verbatim' option and some interesting word choices in your search string, but it's a fairly tedious exercise.

Again, exposing the internals of the Google search algorithm would be interesting here, as it seems clear certain 'authoritative sites' are assigned better scores in the search ranking - not because they have more backlinks or more accurate content, but merely because of governmental and media pressure.

>...Sounds like a government intervention is needed and the monopoly should be broken down for a better functioning market.

You can break up the phone systems because the child companies can provide similar levels of service. But how do you break up a single search algorithm?

How many Larry Page competitors are there? How many Sergey Brins, to listen to an idea without any validation, and judging it with independence? Why do they have to do everything?

Thanks, now I know this 'personalized feed' seen in the Google apps on mobile actually has a product name!

https://developers.google.com/search/docs/advanced/mobile/go...

Some of the content on there is, however, fairly obvious clickbait, but there's also a super easy (working!) 'not interested' button in there.

Ultimately I think it’s because the internet is just too big! Google were only able to do it because they had the right algorithm early in the age of the internet. They were then able to grow with it to achieve the scale required. Starting from scratch now on a general internet search engine would be close to impossible without 10s, if not 100s, billions of investment. And you would need that to build the index before even beginning to be competitive. No one is making bets that big on search, especially when the online advertising industry (which is the only way to fund it currently) is in danger of massive regulation.

I think there is massive opportunity for domain specific search engines though, imagine a search engine specifically designed for software engineers and developers, or one for academic research (not just papers but all online scientific content, news and discussion), or one targeting the arts. I think it’s these verticals that could be incredible.

You then potentially move towards a building “meta” search engines (if your are older than about 35 you will remember these) that work out what you are searching for and uses a domain specific engine.

Edit:

Just to add to this, people who say that “decentralised” search engines are the only way to compete with Google are not completely wrong, it’s just that it’s not about protocols and distributed indexes. It’s about a community of smaller search engines working within specific domains and collaborating (commercially) on meta search engines, prompting people to search on each others engines if it would be better for that search.

We almost need an “Open Search Co-Op” which smaller search engines can join to share technology and refers users to each other.