Maybe this is naive, but I don't understand what Google's impenetrable moat is around search. It's a hard technical problem, but not one with a concrete barrier to entry. And there are lots of really smart people solving really hard technical problems out in the open. On top of that, much of the recent progress in AI is open-source. Surely that could help? I guess indexing at that scale takes a lot of hardware. This…
The main barrier to enter the search market is not tech, it's money: https://0x65.dev/blog/2019-12-22/google-competition-is-just-... And of course there are lots of other issues. For example the Web is very hostile when you are not Google Bot (i.e. there are a lot of big sites that will forbid you from crawling their content, unless you are Google, or Bing).
> there are a lot of big sites that will forbid you from crawling their content, unless you are Google, or Bing
If you're talking about robots.txt, that's just a suggestion. It doesn't hold any actual preventative power.