We can do better than DuckDuckGo • Hacker News Board

Re: We can do better than DuckDuckGo

#31

Post by [deleted] » Tue, Nov 17, 2020, 8:32 PM UTC

metroholografix wrote: post #19

fbelzile wrote: post #14

Earlier quoted context omitted.

> DuckDuckGo is a mirage ... The privacy problems with this claim are many ... good luck verifying ... Okay, can you list just a few? If you're going to make counter-claims like this, you're going to have to provide evidence. Statements like these are not conducive in gaining popular support for increased privacy.

How do you save a search in a non-personally identifiable way? Do you have a human verify the data belonging to each and every search ? Not saving IPs and/or browser data doesn't solve the problem since the search terms themselves can be personally identifiable. How do you verify that DuckDuckGo does -the minimal and ineffective- things they claim to do? They offer no proof. How do you verify that DuckDuckGo does not…

[deleted]

Re: We can do better than DuckDuckGo

#32

Post by pb7 » Tue, Nov 17, 2020, 8:33 PM UTC

bscphil wrote: post #27

metroholografix wrote: post #19

Earlier quoted context omitted.

How do you save a search in a non-personally identifiable way? Do you have a human verify the data belonging to each and every search ? Not saving IPs and/or browser data doesn't solve the problem since the search terms themselves can be personally identifiable. How do you verify that DuckDuckGo does -the minimal and ineffective- things they claim to do? They offer no proof. How do you verify that DuckDuckGo does not…

> How do you save a search in a non-personally identifiable way? Save a sha256 hash of every search for 24 hours. If you see the same hash from >10 distinct IP addresses in a 24 hour period, save the search terms. That's just off the top of my head, I have no reason to think they're doing it exactly like that. The point is that you're claiming that we shouldn't trust DuckDuckGo because you can't think of a way that t…

>How would you verify that for any centralized service, open source or not?

Other centralized (search) services don't have their entire existence depending on this one factor. What is DDG if not alleged privacy? Just use Bing directly.

Re: We can do better than DuckDuckGo

#33

Post by unethical_ban » Tue, Nov 17, 2020, 8:33 PM UTC

metroholografix wrote: post #11

WA9ACE wrote: post #7

Earlier quoted context omitted.

Do you have a search engine that you prefer to use that claims not to store said information that I might try?

The only solution I see is fully distributed/decentralized search. Run your own crawler or be part of a network that distributes this out to each participating node. Every centralized search engine has immensely hard-to-resist and powerful incentives to play "The Eye of Sauron" with your data. Additionally, they offer single points of compromise to other, far more powerful actors. Whatever guarantees DuckDuckGo gives…

Yeah, now you're just saying "Nothing centralized can ever be trusted". So just say that rather than nitpicking their ToS. You weren't going to care what they said anyway.

Re: We can do better than DuckDuckGo

#34

Post by jbay808 » Tue, Nov 17, 2020, 8:34 PM UTC

bscphil wrote: post #21

> they’ve demonstrated gross incompetence in privacy Not sure I buy the example that is given here. 1. It's an issue in their browser app, not their search service. 2. It's not completely indefensible: it allows fetching favicons (potentially) much faster, since they're cached, and they promise that the favicon service is 100% anonymous anyway. 3. They responded to user feedback and switched to fetching favicons loca…

Maybe instead of hard-coding these preferences in the search engine, or having it try to guess for you based on your search history, you can opt-in to download and apply such lists of ranking modifiers to your user profile. Those lists would be maintained by 3rd parties and users, just like eg. adblock blacklists and whitelists. For example, Python devs might maintain a list of search terms and associated urls that get boosted, including stack exchange and their own docs. "Learn python" tutorials would recommend you set up your search preferences for efficient python work, just like they recommend you set up the rest of your workflow. Japanese python devs might have their own list that boosts the official python docs and also whatever the popular local equivalent of stackexchange is in Japan, which gets recommended by the Japanese tutorials. People really into 3D printing can compile their own list for 3D printing hobbyists. You can apply and remove any number of these to your profile at a time.

Re: We can do better than DuckDuckGo

#35

Post by h2onock » Tue, Nov 17, 2020, 8:37 PM UTC

WA9ACE wrote: post #7

metroholografix wrote: post #6

DuckDuckGo is a mirage and should not be used by privacy-conscious folks. Take a look at its terms of service, information collected section: "We also save searches, but again, not in a personally identifiable way, as we do not store IP addresses or unique User agent strings. We use aggregate, non-personal search data to improve things like misspellings." So they save your web searches and claim that they do so in an…

Do you have a search engine that you prefer to use that claims not to store said information that I might try?

I can hand on heart tell you that Mojeek doesn’t and never has. I know this because I work for Mojeek.

Re: We can do better than DuckDuckGo

#36

Post by cptskippy » Tue, Nov 17, 2020, 8:39 PM UTC

lambda_obrien wrote: post #22

Why couldn't several coordinating specialized search engines share their data via something like "charge the downloader" S3 buckets? Then you get an org like StackExchange who could provide indexed data from their site and the algorithms to search the data the most efficiently, GitHub can do the same for their specific zone of speciality, Amazon, etc. Then anyone who wants to use the data can either copy it to their…

So you're proposing Snowflake for search?

Re: We can do better than DuckDuckGo

#37

Post by hobs » Tue, Nov 17, 2020, 8:43 PM UTC

jbay808 wrote: post #34

bscphil wrote: post #21

> they’ve demonstrated gross incompetence in privacy Not sure I buy the example that is given here. 1. It's an issue in their browser app, not their search service. 2. It's not completely indefensible: it allows fetching favicons (potentially) much faster, since they're cached, and they promise that the favicon service is 100% anonymous anyway. 3. They responded to user feedback and switched to fetching favicons loca…

Maybe instead of hard-coding these preferences in the search engine, or having it try to guess for you based on your search history, you can opt-in to download and apply such lists of ranking modifiers to your user profile. Those lists would be maintained by 3rd parties and users, just like eg. adblock blacklists and whitelists. For example, Python devs might maintain a list of search terms and associated urls that g…

Back in the day you'd have webrings - groups of sites that linked each other in clear association.

Re: We can do better than DuckDuckGo

#38

Post by bscphil » Tue, Nov 17, 2020, 8:44 PM UTC

pb7 wrote: post #32

bscphil wrote: post #27

Earlier quoted context omitted.

> How do you save a search in a non-personally identifiable way? Save a sha256 hash of every search for 24 hours. If you see the same hash from >10 distinct IP addresses in a 24 hour period, save the search terms. That's just off the top of my head, I have no reason to think they're doing it exactly like that. The point is that you're claiming that we shouldn't trust DuckDuckGo because you can't think of a way that t…

>How would you verify that for any centralized service, open source or not? Other centralized (search) services don't have their entire existence depending on this one factor. What is DDG if not alleged privacy? Just use Bing directly.

I don't understand that argument at all. What's the threat model?

I think it's entirely reasonable to be in the following posture: I want as much privacy for my web searches as I can reasonably achieve without having to run a search engine myself. I'm willing to trust that search providers are not saving personally identifiable information or passively turning over search data to law enforcement if they claim that they are not in their terms of service.

That's pretty much the use case for DDG. With Bing you know they are violating your privacy. With DDG you have a promise in writing that they are not. It's hard to see how that's not strictly better than what you get from Bing if privacy is among your core desiderata.

Re: We can do better than DuckDuckGo

#39

Post by mech422 » Tue, Nov 17, 2020, 8:45 PM UTC

jbay808 wrote: post #34

bscphil wrote: post #21

> they’ve demonstrated gross incompetence in privacy Not sure I buy the example that is given here. 1. It's an issue in their browser app, not their search service. 2. It's not completely indefensible: it allows fetching favicons (potentially) much faster, since they're cached, and they promise that the favicon service is 100% anonymous anyway. 3. They responded to user feedback and switched to fetching favicons loca…

Maybe instead of hard-coding these preferences in the search engine, or having it try to guess for you based on your search history, you can opt-in to download and apply such lists of ranking modifiers to your user profile. Those lists would be maintained by 3rd parties and users, just like eg. adblock blacklists and whitelists. For example, Python devs might maintain a list of search terms and associated urls that g…

This would be awesome! I'm so tired of google ignoring what I tell it, and trying to 'guess' what I want.

I'd also love to be able to specify I want results from the last year without having to set it everytime.

Re: We can do better than DuckDuckGo

#40

Post by pfarrell » Tue, Nov 17, 2020, 8:45 PM UTC

bscphil wrote: post #27

metroholografix wrote: post #19

Earlier quoted context omitted.

How do you save a search in a non-personally identifiable way? Do you have a human verify the data belonging to each and every search ? Not saving IPs and/or browser data doesn't solve the problem since the search terms themselves can be personally identifiable. How do you verify that DuckDuckGo does -the minimal and ineffective- things they claim to do? They offer no proof. How do you verify that DuckDuckGo does not…

> How do you save a search in a non-personally identifiable way? Save a sha256 hash of every search for 24 hours. If you see the same hash from >10 distinct IP addresses in a 24 hour period, save the search terms. That's just off the top of my head, I have no reason to think they're doing it exactly like that. The point is that you're claiming that we shouldn't trust DuckDuckGo because you can't think of a way that t…

> How would you verify that for any centralized service, open source or not?

I think, technically, some sort of honeypot verification could prove a compromise (i.e. if information that has very little chance of existing naturally in two systems, say a string a guids).

But... I agree with your point. I don't think this is actually feasible or realistic, just technically possible.