Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

news.ycombinator.com

Post reply on HN
Print view View original

111–120 of 492 posts

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#111

Post by [deleted] »

garaetjjte wrote: post #94
gbmatt wrote: post #12

Ha, yes, I've done that at https://gigablast.com/ . The biggest problems now are the following: 1) Too hard to spider the web. Gatekeeper companies like Cloudflare (owned in part by Google) and Cloudfront make it really difficult for upstart search engines to download web pages. 2) Hardware costs are too high. It's much more expensive now to build a large index (50B+ pages) to be competitive. I believe my algorithms…

>Gigablast has teamed up with Imperial Family Companies Associating with that crank (responsible for recent freenode drama) is very off-putting.

[deleted]

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#112

Post by arthur_sav »

jakub_g wrote: post #37

Cliqz wanted to build new search engine but failed. It's just too difficult to operate at that scale and break the existing monopoly of big G. https://www.burda.com/en/news/cliqz-closes-areas-browser-and... https://news.ycombinator.com/item?id=23031520 https://0x65.dev/blog/2019-12-06/building-a-search-engine-fr...

What if we didn't try to replicate google. Smaller and niche search engines would probably work better in this new world of vast information.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#113

Post by arein3 »

sdfjkl wrote: post #78
thoughtstheseus wrote: post #49

Earlier quoted context omitted.

It is. The alternative is scooping everything and using algos to curate. That seems worse imo.

Perhaps vote on results like on Reddit posts? Gets the junk sites down (and out of the index eventually).

Reddit is a heavily gatekeeped community by the mods in regards to specific topics

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#114

Post by hdjjhhvvhga »

gbmatt wrote: post #12

Ha, yes, I've done that at https://gigablast.com/ . The biggest problems now are the following: 1) Too hard to spider the web. Gatekeeper companies like Cloudflare (owned in part by Google) and Cloudfront make it really difficult for upstart search engines to download web pages. 2) Hardware costs are too high. It's much more expensive now to build a large index (50B+ pages) to be competitive. I believe my algorithms…

Regarding the gatekeeper problem: it's a wild guess but maybe if there was a way to involve users by organizing distributed scraping just for the sake of building a decent index, I'm sure many of them would help.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#115

Post by swframe2 »

marginalia_nu wrote: post #63
swframe2 wrote: post #50

Have a look at gpt-3 if you want to see what the future dominant search engine will be. It will not find relevant results, it will write it on the fly customized for exactly what you want to read. (Maybe products will just ship to your door and be auto paid because the future ad targeting AIs will know you so well.)

What if you are looking for something written by a human?

You can always go to a library or bookstore.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#117

Post by rocqua »

maxlamb wrote: post #40
BitwiseFool wrote: post #4

Natural Language Processing is a pox on modern search engines. I suspect that Google et. al. wanted to transform their product into an answer engine that powers voice assistants like Siri and just assumed everyone would naturally like the new way better. I can't stand how Google is always trying to guess what I want, rather than simply returning non-personalized results solely based on exactly what I typed in the tex…

What’s a pox?

Saying X is a pox on Y means saying X is bad for Y.

It originates from the disease 'the pox'.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#118

Post by notriddle »

arein3 wrote: post #113
sdfjkl wrote: post #78

Earlier quoted context omitted.

Perhaps vote on results like on Reddit posts? Gets the junk sites down (and out of the index eventually).

Reddit is a heavily gatekeeped community by the mods in regards to specific topics

Also, vote bots

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#119

Post by marginalia_nu »

swframe2 wrote: post #115
marginalia_nu wrote: post #63

Earlier quoted context omitted.

What if you are looking for something written by a human?

You can always go to a library or bookstore.

Let's imagine I want to talk to the author of the content. How can I do that if it's just a souped up markov chain?

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#120

Post by smt88 »

gbmatt wrote: post #12

Ha, yes, I've done that at https://gigablast.com/ . The biggest problems now are the following: 1) Too hard to spider the web. Gatekeeper companies like Cloudflare (owned in part by Google) and Cloudfront make it really difficult for upstart search engines to download web pages. 2) Hardware costs are too high. It's much more expensive now to build a large index (50B+ pages) to be competitive. I believe my algorithms…

What if you allowed trusted contributors to "donate" their browsing to your index?

AltaVista and Yahoo did that with browser plugins in the 90s.

Post reply on HN