Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

news.ycombinator.com

Post reply on HN
Print view View original

181–190 of 492 posts

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#181

Post by jeffbee »

criddell wrote: post #145
vikingerik wrote: post #73

Earlier quoted context omitted.

Why didn't you just search "austin subaru service"? If you want a query narrowed down by location, that's your job to say so. Sure, it feels great when the engine guesses something like that correctly -- but it comes out worse overall for the plentiful cases where you have to try to compensate for it guessing wrong.

Why should I have to do all that work? I want the machine to do it for me. I can only think of examples where I want personalization. What's an example query where it interferes?

Amazing that the same site that thinks copilot will just generate programs for us also thinks it is literally a crime for a search engine to infer anything.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#182

Post by bla3 »

JohnFen wrote: post #95
Kiro wrote: post #76

Earlier quoted context omitted.

Try searching for the same thing from your computer and your phone and you will get different results. Also, their results come from Bing so any improvement happens at Microsoft.

They do use Bing, but not solely Bing. DDG isn't just a frontend to a different search engine.

It's a bing frontend with a few special cases handled differently. For most queries, you get bing results. Easy to check by comparing results.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#184

Post by jeffbee »

willcipriano wrote: post #179
jeffbee wrote: post #168

Earlier quoted context omitted.

That would be easily the worst search engine ever deployed. Imagine just returning all docs containing the word “bicycle” in chronological order. Useless.

For "Bicycle" it would suck but I don't often use search engines that way, for "High Timber ALX 29" you'd probably get something like this: https://www.schwinnbikes.com/products/high-timber-alx-29?var... I wouldn't use it for everything but sometimes that is the exact behavior that I want. I'd use duck duck go for more general searches.

That is the top hit on google for that search, so what’s your complaint?

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#185

Post by pkamb »

I would use a search engine that only indexed Reddit, Stack Exchange, Wikipedia, and a small number of other sites.

And that specifically blocked Pinterest, Quora, most non-personal “blogs”, etc.

People suggest DDG ! operators, but I don’t want to use a site’s (bad, single-site) search box. I want a multi-site SERP that only displays results from known good sites, which are customizable.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#186

Post by marginalia_nu »

JohnHaugeland wrote: post #173
marginalia_nu wrote: post #147

Earlier quoted context omitted.

Is it not the processing of natural language?

Would you call addition a system of linear equations? No, you don't use the college senior label for the highschool freshman topic. You use the smallest label that fits. It's string processing. NLP is actually understanding the language. Stemming is simple string matching. Playing the technicality game to stretch fields to encompass everything you think even marginally related isn't being thorough or inclusive; it's…

Stemming is a task specific to a natural language. You can't run an English stemmer on French and get good results, for example.

All NLP is, strictly speaking, more or less elaborate string matching.

> Splitting on spaces also isn't NLP.

String splitting can be, but it's a bit borderline. I'll argue you're in NLP territory if it doesn't split "That FBI guy i.e. J. Edgar Hoover." into four "sentences".

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#187

Post by DavidCole1 »

gbmatt wrote: post #12

Ha, yes, I've done that at https://gigablast.com/ . The biggest problems now are the following: 1) Too hard to spider the web. Gatekeeper companies like Cloudflare (owned in part by Google) and Cloudfront make it really difficult for upstart search engines to download web pages. 2) Hardware costs are too high. It's much more expensive now to build a large index (50B+ pages) to be competitive. I believe my algorithms…

Interesting. I had some interests in building a search engine myself (for playing around ofcourse). I had read a blog post by Michael Nielson [1] which had sparked my interest. Do you have any written material about your architecture and stuff like that? Would love to read up.

[1]: https://michaelnielsen.org/ddi/how-to-crawl-a-quarter-billio...

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#188

Post by marginalia_nu »

sdfjkl wrote: post #78
thoughtstheseus wrote: post #49

Earlier quoted context omitted.

It is. The alternative is scooping everything and using algos to curate. That seems worse imo.

Perhaps vote on results like on Reddit posts? Gets the junk sites down (and out of the index eventually).

Given Reddit is notorious for it's problems with astroturfing and vote bots, I don't think this is a particularly promising approach.

Re: Ask HN: Why doesn't anyone create a search engine comparable to 2005 Google?

#189

Post by gbmatt »

pythux wrote: post #171
gbmatt wrote: post #97

Earlier quoted context omitted.

both ddg and brave are bing (microsoft) in disguise.

This is not correct. Brave Search owns its own (growing) index and relies on third-parties like Bing for some fraction of the requests. Which is not the same thing as relying fully on Bing or third-parties for results like so many meta-search engines. More detailed answer here: https://search.brave.com/help/independence Edit: Forgot to say that I work on Brave Search.

brave 'falls back' to bing. which in my experience is most of the time. in fact, out of all the queries i did a while back, they all seemed to come directly from bing. is there a way to disable the reliance on bing and get pure 'brave only' results? and can you be more specific as to what this fraction is? do you blend at all?
Post reply on HN