As I pointed out to paulg yesterday this was
exactly the business model / concept that Blekko was created to address. The idea being that one could use "slashtags" to curate web sites that were "good" on a topic (not spammy) and pull results from that rather than the general web. Guess what? It works great! Also, it doesn't make enough money to support the company using advertising.
For a couple of years, Blekko ran a "3 card monte" game where we white listed the results from Google, Bing, and our own index. For every "contested" query, Blekko consistently beat the others by a significant margin. If the query wasn't contested, Bing and Google did about the same, and if the query was obscure, typically Google did better than Bing or Blekko.
What is a "contested" query? That is one where there is a lot of money on the line. My favorite one was "best credit card" (which is search engine shorthand for "What is the best credit card?" because the stop words "What", "is", and "the" are removed).
Why is it contested? Because if you put an advertisement into the results of that query, and the person making it clicked on that link and signed up for a credit card, you could be paid $50 or more. For a single click. Other queries that advertisers would pay well for getting the traffic of the user were, car dealerships, hotel chains, jewelry retailers, and university "referral" services (like the one that was busted for getting people into Ivy League schools by faking academic records).
Extremely few people click on an ad put onto a page of search results for the query "what is shoe rubber made of?"[1]. However it is required to serve queries like that so that people will come back when they are looking to spend money on something.
So using the same exact idea that Paul proposed Blekko built an English language index which allowed you to curate the crap out of your search results and return much better data. The "value" of that was not considered to be high enough to insist on people logging in to use the engine. Knowing an id for the person making the query allowed for user specific blacklists of spammers (so if for example you never wanted to see a Pinterest link in your results you could make that happen).
Without sufficient traffic, using the feedback loop "of these documents, which one was clicked as the 'best' answer?" type algorithms for ranking fail to converge rapidly enough for decent ranking.
Without a credible threat that if your site is not included in the index, your traffic will be greatly reduced, it is difficult to negotiate with web sites to permit crawling, rather than deny your crawls with the robots.txt file.
Blekko's best customers and most ardent fans? Reference Librarians. Yup, people who needed web search to do their jobs, not to find the movie times for the latest feature. Blekko never did try to create a subscription service, but I think such a service that is somewhere between free and the $$$ of LexisNexis has a shot, at least as a lifestyle business. You still need to get rights to the data and that gets harder and harder.
[1] Okay, bots do, but humans don't