Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 492 posts

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#51

Post by mirker »

gbmatt wrote: post #12

Ha, yes, I've done that at https://gigablast.com/ . The biggest problems now are the following: 1) Too hard to spider the web. Gatekeeper companies like Cloudflare (owned in part by Google) and Cloudfront make it really difficult for upstart search engines to download web pages. 2) Hardware costs are too high. It's much more expensive now to build a large index (50B+ pages) to be competitive. I believe my algorithms…

If you have customers, does that mean the incremental gain from an improved index costs too much to store? Or are you talking about computational costs?

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#52

Post by technobabbler »

thoughtstheseus wrote: post #31
gbmatt wrote: post #12

Ha, yes, I've done that at https://gigablast.com/ . The biggest problems now are the following: 1) Too hard to spider the web. Gatekeeper companies like Cloudflare (owned in part by Google) and Cloudfront make it really difficult for upstart search engines to download web pages. 2) Hardware costs are too high. It's much more expensive now to build a large index (50B+ pages) to be competitive. I believe my algorithms…

Perhaps trolling the entire web is not useful today? I’d love a search engine where I can whitelist sites or take an existing whitelist from trusted curators.

That's a great idea.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#53

Post by marginalia_nu »

BitwiseFool wrote: post #24
marginalia_nu wrote: post #17

Earlier quoted context omitted.

I think some NLP is strictly beneficial for a search engine. You may think "grep for the web" sounds like a good idea, but let me tell you, having tried this, manually going through every permutation of plural forms of words and manually iterating the order of words to find a result is a chore and a half. Like, instead of trying PDP11 emulator PDP-11 emulator "PDP 11" emulator PDP11 emulators PDP-11 emulators "PDP 11…

Maybe I'm not using the right qualifiers around the term NLP. The kind of NLP I was referring to is something like "Hey google, what is natural language processing?" and orienting the search around people asking questions in standard(ish) English like they would to another person.

NLP is very heavily integrated into search, so I don't think it's really possible to decouple them. But I agree the whole BonziBuddy thing they've got going now is annoying and it's especially unfortunate how it's replaced the search functionality. I'd have a lot more patience with it if I could choose this functionality when I wanted to ask a question.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#54

Post by afrcnc »

gbmatt wrote: post #12

Ha, yes, I've done that at https://gigablast.com/ . The biggest problems now are the following: 1) Too hard to spider the web. Gatekeeper companies like Cloudflare (owned in part by Google) and Cloudfront make it really difficult for upstart search engines to download web pages. 2) Hardware costs are too high. It's much more expensive now to build a large index (50B+ pages) to be competitive. I believe my algorithms…

how recent are your results? 1-2h? 1 day?

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#55

Post by andrewclunn »

What about a search engine that only indexed information and technology "alternative" sites, specifically to give you the results most likely to be purged or demoted from Google's results? Would be simple enough in scope and have a built in market and use.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#56

Post by rasengan »

ZetaZero wrote: post #11
Lucasoato wrote: post #3

Earlier quoted context omitted.

Tried it but it just says: "Something went wrong. Please try again."

same here

It should be working now! Thanks for the heads up. There was a traffic issue.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#58

Post by RDaneel0livaw »

BbzzbB wrote: post #22

No mention of DDG in the comments? Is there a reason I'm not seeing or it's just not the preferred alt-search on HN? Seems to have been working fine for me when I struggle to get past the funnels and content mills on Google.

I was looking for this as well! I use it daily and have for years. Love it.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#59

Post by nfriedly »

I think DuckDuckGo is closer to what you want. Same results for everyone, better privacy, and they're proactive about improving their results.

https://duckduckgo.com/

Part of the problem is that there's a lot more low-quality content to wade through now than there was in 2005. I think the Google of 2005 would have trouble delivering quality results today also.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#60

Post by marginalia_nu »

KennyBlanken wrote: post #41
marginalia_nu wrote: post #17

Earlier quoted context omitted.

I think some NLP is strictly beneficial for a search engine. You may think "grep for the web" sounds like a good idea, but let me tell you, having tried this, manually going through every permutation of plural forms of words and manually iterating the order of words to find a result is a chore and a half. Like, instead of trying PDP11 emulator PDP-11 emulator "PDP 11" emulator PDP11 emulators PDP-11 emulators "PDP 11…

Google does go way overboard with "NLP". Starting at least 5 years ago there was a trend toward "similar" matching and search result quality nose-dived. You can search for, say, "cycling (insert product category here)" and get motorcycle related results. Why? Because to google "cycling" = "biking" and "motorcycles" are "bikes", bob's your uncle, now you're getting hits for motorcycle products. Every time I try to do…

Yeah. It's one of those things where it's invisible where it works and enraging when it doesn't. That's generally not a failure mode that's desirable. It at least should require extremely low failure rates to motivate.
Post reply on HN