Live data from Hacker News

We can do better than DuckDuckGo

drewdevault.com

101–110 of 383 posts

Re: We can do better than DuckDuckGo

#102
post #52

DDG does operate their own crawler[1], though they also do still rely on third parties[2]. [1] https://help.duckduckgo.com/duckduckgo-help-pages/results/du... [2] https://help.duckduckgo.com/duckduckgo-help-pages/results/so...

Author didn't even DDG to find this out?

Clearly neither did you as he is correct. DDGs crawler is not a crawler like googlebot.

Re: We can do better than DuckDuckGo

#103
post #95
post #21

> they’ve demonstrated gross incompetence in privacy Not sure I buy the example that is given here. 1. It's an issue in their browser app, not their search service. 2. It's not completely indefensible: it allows fetching favicons (potentially) much faster, since they're cached, and they promise that the favicon service is 100% anonymous anyway. 3. They responded to user feedback and switched to fetching favicons loca…

I thought DDG already crawled their own curated list of sites? There is a DuckDuckGoBot and I think it was an interview or podcast Gabriel did a while back that he mentioned they use it for filling out gaps in the Bing API data to provide the instant answers, favicons. Their preference for the instant answers were authoritative references such as docs.python.org. This would have been a while back though.

If memory serves, those crawls are only used for Instant Answers. My interpretation of the blog post is that it would be nice to have a search engine that's sort of a hybrid approach based on Instant Answers for the whole web.

Re: We can do better than DuckDuckGo

#104

Am I the only person who just doesn't have problems with DDG search results? What am I doing wrong (or right), here? I put a thing in and find it. I just don't use Google any more. Genuinely curious why it's working for me and such garbage for everyone else.

for generic stuff DDG is mostly ok. But for local results, even though it has a switch for local results, it REALLY REALLY REALLY sucks bad and often doesn't get any of the expected places anywhere in the first few pages for New Zealand which makes it somewhat useless

Re: We can do better than DuckDuckGo

#105
I wonder if instead of another search engine we would benefit from a directory, like DMOZ, or perhaps something tag based or non-hierarchical. Sometimes I find better results by first finding a good website in space of my query, and then searching within that site, as opposed to applying a specific query over all websites. Once example would be recipes: if you search for "bean burger recipe" you will get lots of results across many website, but some may not be very good, whereas if you already know of recipe websites that you consider high-quality or match your preferences, then you'll find the best (subjectively) recipe by visiting that site and then searching for bean burgers.

Re: We can do better than DuckDuckGo

#106
post #21

> they’ve demonstrated gross incompetence in privacy Not sure I buy the example that is given here. 1. It's an issue in their browser app, not their search service. 2. It's not completely indefensible: it allows fetching favicons (potentially) much faster, since they're cached, and they promise that the favicon service is 100% anonymous anyway. 3. They responded to user feedback and switched to fetching favicons loca…

> and they promise that the favicon service is 100% anonymous anyway.

With that logic, Apple’s OCSP server is also 100% anonymous (which I legitimately can believe it is).

Re: We can do better than DuckDuckGo

#107
post #34
post #21

> they’ve demonstrated gross incompetence in privacy Not sure I buy the example that is given here. 1. It's an issue in their browser app, not their search service. 2. It's not completely indefensible: it allows fetching favicons (potentially) much faster, since they're cached, and they promise that the favicon service is 100% anonymous anyway. 3. They responded to user feedback and switched to fetching favicons loca…

Maybe instead of hard-coding these preferences in the search engine, or having it try to guess for you based on your search history, you can opt-in to download and apply such lists of ranking modifiers to your user profile. Those lists would be maintained by 3rd parties and users, just like eg. adblock blacklists and whitelists. For example, Python devs might maintain a list of search terms and associated urls that g…

This doesn't really seem immune from spam.

I got signed up for goodreads (book review site), and I get tons of spam. It's not quite the same as your idea, but it is a currated list. I don't know how you stop spammers from adding bogus links in the python interest list (to use an example).

This is a hard problem..

EDIT: Clarified goodreads reference!

Re: We can do better than DuckDuckGo

#109
How about a crowdsourced search engine like wikipedia or stackoverflow? Like:

When you search for "kittens" you get the links that are most upvoted by the community.

If nobody has ever submitted links for search term "kittens" , you get a link to selected generic search engines. And "kittens" end up into a list of words someone has searched but nobody has yet added a good result link for.

Re: We can do better than DuckDuckGo

#110
>We need a real, working FOSS search engine, complete with its own crawler.

How would an open-source search engine stand against abusive SEO-optimization? If anyone can understand how the ranking algorithm works then anyone can game it.

Post reply on HN