Live data from Hacker News

Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

news.ycombinator.com

441–450 of 492 posts

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#441
post #231

Earlier quoted context omitted.

A subscriber-supported search engine sounds cool to me. Any precedent?

Kagi.com does this. In closed beta at the moment, but you can email and request access.

I've tested Kagi a bit. It nicely gave me exactly what I wanted even in cases where names could have different meanings in different contexts (I tested with Kotlin)

The basic results are good with some nice touches here and there like including a "blast from the past" section with older results which is actually what I sometimes want and another section where it widens up a bit (i.e. what Google does by default?).

Furthermore you can apply pre defined search "lenses" that focuses your search, or even make your own, and you can boost or de-rank sites.

I had not expected this to happen so quickly but I'm going to move from DDG to kagi as my default search engine for at least a couple of days because I am fed up with both Googles and DDGs inability to actually respect my queries.

If ir continues to work as well as it does today I'll happily pay $10 a month and I might also buy 6 months gift cards for close friends and family for next Christmas.

Think about it, unlike an ad financed engine incentives are extremely closely aligned here: the smartest thing Kagi can do is to get my results as fast as possible to conserve server resources (and delight their customer).

For an ad financed engine abd especially one that also serves ads on search results pages as well the obvious thing to do is to keep me bouncing between tweaking my search query and various that almost has my question answered but not quite.

(That said, if one us going to stay mainstream I recommend DDG over Google since 1. for me at least Googles results are just as bad and 2. with DDG it is at least extremely easy to check with Google as well to see if they have a better result 3. competition is good)

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#442
All big tech businesses at their core are monopolies. Once a significant field has been figured out, it is very difficult to compete with the market standard, unless they screw up so hard that that THE AVERAGE user starts searching for an alternative.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#443
post #433

Earlier quoted context omitted.

> Hardware costs are too high I want to say - you don’t know what are talking about. But, it’ll be rude. Hardware is much cheaper and powerful now compared to 2005.

the complexity of the search algorithm has also increased substantially since 2005 And, in 2005, a billion page index was pretty big. Now it's closer to 100 billion.

There were ~60B pages on Facebook in 2015 I think your numbers are outdated. - Google search SRE

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#444
post #16

Early 2000s google index ran in a garage. The current google index has dedicated power stations. It's a bit like the car industry - you could run a startup from your garage in the early days but you need titanic amounts of capital to compete now thanks to vertical integration. Major governments and billionaires can compete but everybody else is locked out of the market (most "startups" use bings index).

Google's datacenters are huge because they save user behavior data, not because their web search index is particularly big. Also, Google Search wastes a lot of resources on the "search as you type" feature.

Running a search engine in your garage is feasible today because hardware and connectivity have improved much faster than the size of the WWW.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#445

Some people try: https://www.mojeek.com/ https://fireball.com/ https://search.brave.com/

Mojeek founder story here: https://blog.mojeek.com/2021/03/to-track-or-not-to-track.htm... No-tracking and independent from the start. Now at 4.6 billion pages with own infrastructure and IP. Went to market in 2020 with contextual ads and API. Self-disclosure: CEO

For a fully independent indexer, probably the best results I have seen so far. For me the minimum baseline is searching for "444" and if it doesn't return 444.hu as the first result, its a no-go.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#447
post #254

Earlier quoted context omitted.

Mojeek founder story here: https://blog.mojeek.com/2021/03/to-track-or-not-to-track.htm... No-tracking and independent from the start. Now at 4.6 billion pages with own infrastructure and IP. Went to market in 2020 with contextual ads and API. Self-disclosure: CEO

Do you use some sort of PageRank?

Yes, something conceptually similar to PageRank but our own thing which we call Gravity.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#448

I'd like a way of automatically filtering for websites that : * Don't use JS * Don't use Google analytics * Don't weigh more than a few kB per page * Don't show any sites with ads That would be a place to begin.

Almost all websites use JavaScript, including this one.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#450
post #240

Earlier quoted context omitted.

> 2) Hardware costs are too high. Which is why the next big search engine should be distributed: https://yacy.net .

No way to test it right away, demo peer 502-es.

You could search for other public-facing instances, e.g., http://sokrates.homeunix.net:6060.
Post reply on HN