We can do better than DuckDuckGo

#1

Post by als0 » Tue, Nov 17, 2020, 6:16 PM UTC

We can do better than DuckDuckGo
drewdevault.com

Re: We can do better than DuckDuckGo

#2

Post by suff » Tue, Nov 17, 2020, 6:16 PM UTC

No you can't.

Re: We can do better than DuckDuckGo

#3

Post by suff » Tue, Nov 17, 2020, 6:18 PM UTC

I take that back. Aside from loads of money and boundless dev time, you've got it all figured out :-)

Re: We can do better than DuckDuckGo

#4

Post by andreareina » Tue, Nov 17, 2020, 6:31 PM UTC

DDG does operate their own crawler[1], though they also do still rely on third parties[2].

[1] https://help.duckduckgo.com/duckduckgo-help-pages/results/du...

[2] https://help.duckduckgo.com/duckduckgo-help-pages/results/so...

Re: We can do better than DuckDuckGo

#5

Post by nikivi » Tue, Nov 17, 2020, 7:54 PM UTC

I'd love a truly open source world class search engine. Curious how both the crawler and the search index / search is done by the likes of Google/Bing/DDG. Eventually someone will make an oss version of it that can compete.

The beauty of such oss solution maybe the custom heuristics that can be created based off the crawled data.

Re: We can do better than DuckDuckGo

#6

Post by metroholografix » Tue, Nov 17, 2020, 7:54 PM UTC

DuckDuckGo is a mirage and should not be used by privacy-conscious folks. Take a look at its terms of service, information collected section:

"We also save searches, but again, not in a personally identifiable way, as we do not store IP addresses or unique User agent strings. We use aggregate, non-personal search data to improve things like misspellings."

So they save your web searches and claim that they do so in an non-personally identifiable way. The privacy problems with this claim are many, even if one accepts it at face value (good luck verifying that this is the case).

Re: We can do better than DuckDuckGo

#7

Post by WA9ACE » Tue, Nov 17, 2020, 8:00 PM UTC

metroholografix wrote: post #6

DuckDuckGo is a mirage and should not be used by privacy-conscious folks. Take a look at its terms of service, information collected section: "We also save searches, but again, not in a personally identifiable way, as we do not store IP addresses or unique User agent strings. We use aggregate, non-personal search data to improve things like misspellings." So they save your web searches and claim that they do so in an…

Do you have a search engine that you prefer to use that claims not to store said information that I might try?

Re: We can do better than DuckDuckGo

#8

Post by ddorian43 » Tue, Nov 17, 2020, 8:01 PM UTC

nikivi wrote: post #5

I'd love a truly open source world class search engine. Curious how both the crawler and the search index / search is done by the likes of Google/Bing/DDG. Eventually someone will make an oss version of it that can compete. The beauty of such oss solution maybe the custom heuristics that can be created based off the crawled data.

Good luck with that mate. Check out https://commoncrawl.org/

Re: We can do better than DuckDuckGo

#9

Post by pmoriarty » Tue, Nov 17, 2020, 8:02 PM UTC

My main problem with DDG is that there's no way to be sure they actually respect their users' privacy as they claim to.

Ideally, services like theirs would be continuously audited by respectable, trusted organizations like the EFF.. multiple such organizations even.

Then I'd have at least some reason to believe their claims of not collecting data about me.

As it stands, I only have their word for it.. which in this day and age is pretty worthless.

That said, I'd still much rather use DDG, who at least pay lip service to privacy, than sites like Google or Facebook, who are openly contemptuous of it.

At the very least it sends a message to these organizations that privacy is still valued, and they'd lose out by not trying to accommodate the privacy needs of their users to some extent.

Re: We can do better than DuckDuckGo

#10

Post by Mediterraneo10 » Tue, Nov 17, 2020, 8:06 PM UTC

nikivi wrote: post #5

I'd love a truly open source world class search engine. Curious how both the crawler and the search index / search is done by the likes of Google/Bing/DDG. Eventually someone will make an oss version of it that can compete. The beauty of such oss solution maybe the custom heuristics that can be created based off the crawled data.

The challenges to OSS developers are numerous. First of all, many popular sites on the internet block crawlers other than Google and Bing, because only those ones seem to matter to their business, and any small upstart would be assumed to be a dodgy bot. Secondly, Google amasses the database it has only with vast data centers, incredible amounts of bandwidth, and power requirements unavailable to a startup.