DuckDuckGo: Escape your search engine filter bubble • Hacker News Board

Re: DuckDuckGo: Escape your search engine filter bubble

#51

Post by yanw » Mon, Jun 20, 2011, 2:06 PM UTC

Revisor wrote: post #48

yanw wrote: post #35

Shame on DDG for endorsing this nonsense. I understand the quest for a larger userbase but please don't use this sort of FUD that is being peddled by those who don't understand the technical aspects of search and are trying to sell their books. A search engine's prime function is to filter the millions of results for each query down to the most relevant results for each individual user, and never the same 10 results…

Search queries have different purposes. Personalization based on my preferences and past search works wonders for known-item seeking and for re-finding. But for exploratory search and exhaustive search personalization works as an echo chamber (or filter bubble if you will) . Then there is the problem of change and inertia. People change, their preferences vary in time. Personalization has an inertia that causes the r…

All can be remedied by tweaking the algorithms and that happens all the time, you can also use a certain vertical directly for certain type queries (scholar search etc).

The same way that what a user might enjoy can be inferred the inverse of that can be used as a signal in the algorithm as well. Personalization is just relevance and all is just a matter of tuning.

Re: DuckDuckGo: Escape your search engine filter bubble

#52

Post by rcamera » Mon, Jun 20, 2011, 2:07 PM UTC

ThomPete wrote: post #5

It is the ultimate quest for serendipity. If you filter by what you want you don't get what you didn't know you wanted. I will give DDG the benefit of the doubt and try it out for a couple of months. Starting now.

I'm glad I have been using DDG for over 6 months now. For the new user, here are some advices.

DDG is my default search engine. Basically, all of my searches are completed in DDG, only very few times I have to go to google to find what I am looking for. My suggestion is that you spend 5 minutes learning the bang! syntax, as it speeds up your search by alot, then set DDG as your default search engine for a couple weeks. You will see that the bang! syntax and the 0-click-info really makes the difference on search speed, and their results are pretty damn good.

Re: DuckDuckGo: Escape your search engine filter bubble

#53

Post by mkr-hn » Mon, Jun 20, 2011, 2:08 PM UTC

speckledjim wrote: post #45

yanw wrote: post #35

Shame on DDG for endorsing this nonsense. I understand the quest for a larger userbase but please don't use this sort of FUD that is being peddled by those who don't understand the technical aspects of search and are trying to sell their books. A search engine's prime function is to filter the millions of results for each query down to the most relevant results for each individual user, and never the same 10 results…

Spreading FUD about tracking search terms etc didn't seem to help DDG last time. I doubt this will have any effect either. DDG: If you want to grow, don't keep attacking the big search engines with FUD that only a miniscule % of people care about.

The February DDG newsletter reported growth in January from 2.5m searches to 5m. I don't think DDG needs FUD to grow.

Re: DuckDuckGo: Escape your search engine filter bubble

#54

Post by dhimes » Mon, Jun 20, 2011, 2:09 PM UTC

yanw wrote: post #19

retube wrote: post #11

Yeah this is an important issue. Google apparently ranks results (for you personally) based on some 57 inputs, even when not logged into Google services. In short results suffer from a self-reinforcing feedback loop, forever constraining what you see. I wonder how easy it is to get "clean" or "default" results from Google? And I know it's been discussed many times , but just how easy would it be to maintain real anon…

No it's a non-issue, and by the way Google probably uses much more that 57 'inputs' (signals) to determine relevance. Personalization is just a different term for relevant results and this is just FUD, the source of which is people who don't understand the technical aspects.

I would agree with you if there was a button I could click to turn it off. Their algorithms aren't infallible by any means. Plus, ultimately they are a business and trying to sell, so they are going to eventually skew results towards things they think I may be willing to buy, rather than things I want to know. I think G is more susceptible than MS, but only because MS does make a living selling other stuff.

I'm not saying it's evil or wrong, just that if my first results don't appeal to me, a simple measure that might work for me would be to turn the filter off. Anyway, they could learn more about me if they let me do that.

Re: DuckDuckGo: Escape your search engine filter bubble

#55

Post by hollerith » Mon, Jun 20, 2011, 2:12 PM UTC

I heard you can add "&pws=0" to a search query to turn off personalization:

http://google.com/search?q=paul+graham&pws=0

Re: DuckDuckGo: Escape your search engine filter bubble

#56

Post by jemfinch » Mon, Jun 20, 2011, 2:12 PM UTC

loup-vaillant wrote: post #26

Confirmation bias is bad enough already, it's really a shame that powerful companies like Google reinforces it just for the sake of giving you more pleasing search results. (Pleasing and Good overlap, but they're not equal.) http://en.wikipedia.org/wiki/Confirmation_bias http://lesswrong.com/lw/iw/positive_bias_look_into_the_dark/ Edit: I didn't intend to bash Google specifically. But they are faced with a choice in…

This sort of personalization is no different than using a user's click history to determine whether he means cycling or motorcycling when he searches for "biking". It's no different than using history to determine whether someone is looking for a television schedule or coding help when he searches for "programming".

One person's "confirmation bias" is another person's "relevance". Increasing relevance to the user will naturally result in confirming their biases, because there's (probably) no programmatic way to discern contentious subjects in which confirmation bias is applicable from non-contentious subjects where it's not.

The only "shame" I see here is people who ascribe some sort of devious intention to what's clearly the natural result of trying to solve the most important problem in search.

Re: DuckDuckGo: Escape your search engine filter bubble

#57

Post by quinndupont » Mon, Jun 20, 2011, 2:13 PM UTC

yanw wrote: post #35

Shame on DDG for endorsing this nonsense. I understand the quest for a larger userbase but please don't use this sort of FUD that is being peddled by those who don't understand the technical aspects of search and are trying to sell their books. A search engine's prime function is to filter the millions of results for each query down to the most relevant results for each individual user, and never the same 10 results…

I think you missed the point. Yes DDG ranks, but it may rank differently. Of course, there is an implicit slam on them here: instead of showing you what you want the vast majority of the time, they show you other stuff. This is good on occasion (so you don't live in an echo chamber), but if this was all the time it would mean lower quality search results for simple things.

Re: DuckDuckGo: Escape your search engine filter bubble

#58

Post by qjz » Mon, Jun 20, 2011, 2:13 PM UTC

igorgue wrote: post #21

I heard about this book on NPR a couple of weeks ago [1]. I really don't like this biased view of ML. In summary what he said on the show was something like: At least the news shows (and news pappers, radio) gives the same information to everyone, so instead of showing Kardashians news they do show you Bin Laden news, even though, they know the Kardashians are more profitable... this is not the case with Google Searc…

I think there's reason to complain that the reality is far from the ideal. Take Netflix, for example. I rated hundreds of movies, filled out the taste preferences to narrow down the genres I was most interested in, but the "Suggestions for You" were underwhelming, to say the least. Even worse, they simply never changed. So I went back into my taste preferences and checked that I "Often" watched every single mood and genre listed. Now I get a vastly improved range of suggestions, exposing me to some great movies, simply because Netflix is no longer hiding them from my view. I may have a unique individual taste for movies, but Netflix sure hasn't fathomed my criteria with their algorithm.

Re: DuckDuckGo: Escape your search engine filter bubble

#59

Post by Kylekramer » Mon, Jun 20, 2011, 2:14 PM UTC

It is well known that people seek out information that reinforces their own beliefs. And while it may not be ideal for an enlightened population, why should Google/Bing fight this natural inclination? Their job is to ultimately give their users what they are searching for, not challenge their users' opinions. Especially since there are tons of benefits to personalized search, and it may be hard to not throw the baby out with the bathwater.

Re: DuckDuckGo: Escape your search engine filter bubble

#60

Post by widget » Mon, Jun 20, 2011, 2:16 PM UTC

Any filter at all will result in a "Filter Bubble" as defined here (except, I suppose, returning a randomly sorted list of all sites on the internet). Whether the filter is personalized or not doesn't change that fact. What is the actual benefit of everyone getting the same search results when looking for a particular term? Search engines are not news sites or research papers: biasing towards relevant results is not a bad thing.