Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We can do better than DuckDuckGo

drewdevault.com

Post reply on HN
Print view View original

271–280 of 383 posts

Re: We can do better than DuckDuckGo

#271

Post by DenverCode »

jimnotgym wrote: post #207

> The search results suck Do they really though, for normal people that is?. Some of my searches today below, can't remember the exact terms I used. Mix of DDG and Google. 1) Walt Whitman, I wanted a basic overview of his work to satisfy some idle curiosity. DDG gave me his wikipedia page. Bingo 2) EAN-13 check digit. First result wikipedia telling me how to calculate it. I see it is simple and I have a long list in…

I also have no issues with DDG but most of my searches are pretty specific or I'll just end up at the Wikipedia page anyway.

Other than that, I've been using [Runaroo](https://www.runnaroo.com) as well.

Re: We can do better than DuckDuckGo

#272

Post by austinprete »

ddingus wrote: post #251
carom wrote: post #231

Earlier quoted context omitted.

The biggest habit I had to break moving from Google to DDG was phrasing everything as a question. If anyone is thinking of making the switch, you can always redirect your searches to Google by throwing a g! in the query.

Funny, I never used questions with Google, until very recently, and only with some queries. Got good at including words for context early on and never stopped.

Out of curiosity — what made you start?

Re: We can do better than DuckDuckGo

#273

Post by nl »

qiqitori wrote: post #270
nl wrote: post #220

Earlier quoted context omitted.

In consumer search there is a really long tail of questions (in 2017 15% of Google's daily queries have never been seen before[1]) and performance on this is very important. I just searched for "lockdown rules for SA" (I'm in South Australia and we just had a new 20 person cluster, so we are going back into lockdown). On DDG the first results was a Guardian article which was good, but then the rest were a mix of Sout…

Hrm, I think it's extremely iffy to abbreviate South Australia like that in a search query. You don't need the "for" either. BTW, when I perform the same search, Google's first result is "What Are the Lockdown Rules for South Africa? A Guide for ..." and all the other results on the first page are about South Africa too. (Note: I'm in Japan)

> Hrm, I think it's extremely iffy to abbreviate South Australia like that in a search query.

Everyone in Australia uses "SA" - this is one of the reasons why location based context is important.

> You don't need the "for" either.

I worked on consumer search for a few years, and on text based search word like "for" are helpful to get exact match. Even if the term frequency of "for" on its own isn't particularly useful "for SA" absolutely is. (And these days with neural ranking using sub-word parts it is even more useful).

Re: We can do better than DuckDuckGo

#275

Post by ddingus »

nl wrote: post #268
ddingus wrote: post #229

Earlier quoted context omitted.

I got this: https://www.enca.com/news/sa-lockdown-strick-international-t...

Which is about South Africa - which might be a good result for you, depending on where you live. So at least they are trying to to location based results.

Well, I am in PDX, so....

Re: We can do better than DuckDuckGo

#276

Post by ddingus »

austinprete wrote: post #272
ddingus wrote: post #251

Earlier quoted context omitted.

Funny, I never used questions with Google, until very recently, and only with some queries. Got good at including words for context early on and never stopped.

Out of curiosity — what made you start?

Seeing a few queries others did. Now, if it seems like an obvious question, I may try it.

My default remains word searches. Frankly, better operators would benefit me more than questions would.

I really do not always want to formulate a question. Doing that makes sense sometimes.

Often, I want to see relevant info, then formulate other queries.

Re: We can do better than DuckDuckGo

#278

Post by bscphil »

retsibsi wrote: post #177
bscphil wrote: post #47

Earlier quoted context omitted.

I like this idea! I think the biggest difficulty with it - which is also probably the most important reason that engines like Google and DDG are currently struggling to return good results - is that the search space is just so enormously large now. The advantage of the suggestion in the blog post is that you trim down the possible results to a handful of "known good" sources. As I understand it, you'd want to continu…

This may be a very dumb question, but could the filtering be done client-side? As in, DDG's servers do their thing as normal and return the results, then code is executed on your machine to weight/prune the results according to your preferences. Maybe this would require too much data to be sent to the client, compared to the usual case where it only needs a page of results at a time. If so, would a compromise be viab…

This would work if you had a blacklist of domains you didn't want to see. But the idea in the post is closer to a whitelist: the highest priority sites (tier 1) should be set manually, and anything after that should be weighted by how often it's referenced by the tier 1 sites. For a lot of searches you're going to have to pull many results to fill a page with stuff from a small handful of domains, and in fact you might not be able to get them at all. And that's before you start dealing with the weighting issue, which would require quite a bit of metadata to be sent with each request.

Re: We can do better than DuckDuckGo

#279

Post by edgyquant »

chris_f wrote: post #241
bscphil wrote: post #179

Earlier quoted context omitted.

> If I want my blog to show up on your search engine, do I have to get it linked by one of those sites, or can I register with you? Will I be tier 1, or I think what I'd say in defense is that we've misunderstood what search engines are useful for. They're really bad at helping us discover new things. Your blog might be awesome, but it's not going to be easy for a search engine to tell that it's awesome. It's going t…

> For example this is the #2 result for "python f strings" on DDG[1]. It's total garbage, and, quoting the blog, "we can do better". (This result is also on page 1 for the same query on Google.) There are other options that may be better, but in general, very few people are looking for them. Here's your "python f strings" query on Runnaroo: https://www.runnaroo.com/search?term=python+f+strings I'm the creator of Runn…

I tried this out real quick just to see for myself. First it was slow for me, if I wasn’t wanting to try it out I may have closed the tab, but I did a search for a question I recently googled (that was an example of an annoying google search.) I searched “Python arcade sprite opacity“ and while the first three results were the same as google the fourth was a github link to the project itself and brought me to the answer (which wasn’t on page 1 of google for me, although it was on page two.)

So you need to speed it up, but it does look good for searching documentation at least.

Re: We can do better than DuckDuckGo

#280

Post by edgyquant »

Shared404 wrote: post #205
RileyJames wrote: post #126

Earlier quoted context omitted.

Great idea, but why build a search engine at all in this case? You can use DDG + your filter and see only the results from your whitelist. Could easily be implemented for any current search engine. To a large extent, this is what you already do when you view a page of search results. Filter them based on your understand of what sites / results hold value.

> why build a search engine at all in this case? On a public scale, you could make an argument for tighter integration/better privacy with the lists. For example: Browser -----Request-to-SE-----> Search Engine ^ | | Unfiltered Results (In YAML/JSON) | | | V |--Desired Results------ Local Filtering/Rendering On a private scale, if you are only crawling sites on the allow list than you have the possibility of being abl…

Aside from this a big reason to build this is it seems a lot simpler than writing a giant web crawler ala google and thus is a good target for an open source solution. Which is the biggest problem with duck duck go.
Post reply on HN