Earlier quoted context omitted.
See, the problem there is the "paying" part. What do you, the customer, allegedly willing to pay, get from a search provider? Especially once everyone else piles on? What you get is a simple tool, that then requorements bloats as soon as the rest of the economy notices you're a growing centralized control point. You start getting DMCA pipelines. You start getting hosting amd analytics, and monetization. You get your…
So what do I use if I don't want to use google? Or any other free service that's co opted
Each and every internet staple started as a bunch of folks doing a project. Keep an eye out for those, or become one of them. Do it for you. Nobody can tell you not to, or hold you to task for doing so. It's only when you start building up enough network effect driven inertia that you start to become that leverage point ripe for the co-opting.
Knowledge asymmetry is big business. The only way to keep people from pulling the wool over your eyes is to build what you need and have a good ole look yourself.
Be warned it isn't for the light of heart, hard-drive, or net link, and your results are only as good as your crawler.
There are also a lot of risks by perniciously going out and connecting with everything under the sun. I'd recommend keeping your forays constrained, or at least on a well insulated network.