Popular Pirate Sites ‘Disappear’ from DuckDuckGo’s Top Search Results • Hacker News Board

Re: Popular Pirate Sites ‘Disappear’ from DuckDuckGo’s Top Search Results

#121

Post by curryst » Mon, Nov 16, 2020, 7:19 AM UTC

spurgu wrote: post #53

sgift wrote: post #44

Earlier quoted context omitted.

Of course they are. That's the reason Google "won" in the first place after all. They had a algorithm which better fit what people expected.

But tides are turning, Google's results are steadily going downhill. Unfortunately it seems to be too much of a daunting task with today's size of the internet to jump in with a viable competitor.

> But tides are turning, Google's results are steadily going downhill. Unfortunately it seems to be too much of a daunting task with today's size of the internet to jump in with a viable competitor.

I wonder what portion of the internet you have to scrape to be able to have good results for 90% of queries. An enormous number of my queries end up on Wikipedia somewhere, for example. A lot end up on Stack Overflow. The two of those probably constitute the majority of my searches, and almost certainly a majority of the searches where I really care about the answer (and I'm not just googling whether pirates and ninjas were contemporaries of each other, because why not).

If Google's results are really slipping that much, you can make up for the lack of data by providing better results on the data you do have.

I am curious why something like that hasn't gotten more popular. A search engine that only indexes highly reputable and public sources would be interesting. Wikipedia, dictionaries from somewhere, reputable newspapers globally, etc. Is it just too big of a target for abuse to be viable?

Re: Popular Pirate Sites ‘Disappear’ from DuckDuckGo’s Top Search Results

#122

Post by spurgu » Mon, Nov 16, 2020, 5:17 PM UTC

curryst wrote: post #121

spurgu wrote: post #53

Earlier quoted context omitted.

But tides are turning, Google's results are steadily going downhill. Unfortunately it seems to be too much of a daunting task with today's size of the internet to jump in with a viable competitor.

> But tides are turning, Google's results are steadily going downhill. Unfortunately it seems to be too much of a daunting task with today's size of the internet to jump in with a viable competitor. I wonder what portion of the internet you have to scrape to be able to have good results for 90% of queries. An enormous number of my queries end up on Wikipedia somewhere, for example. A lot end up on Stack Overflow. The…

Good points.

> I wonder what portion of the internet you have to scrape to be able to have good results for 90% of queries.

But this is it. I don't think it'd be too hard to get 90% of queries right. It's the 10% that are the challenge.

"Page rank" / "link juice" (amount/strength of incoming links) is still a main ingredient in Google's recipe, so they are actually able to determine, without manual intervention (more or less), which sites are "reputable", while your suggestion would require someone to curate the assortments of "reputable" sites.

How would you discover niche blogs? Or even more high profile ones, just not known to the major public?

That said, as you say, you could probably do with something like this for most of your day-to-day searches, and then just !g or whatever for the rest.

Re: Popular Pirate Sites ‘Disappear’ from DuckDuckGo’s Top Search Results

#123

Post by nvrspyx » Tue, Nov 17, 2020, 7:05 PM UTC

pythux wrote: post #80

nvrspyx wrote: post #15

Earlier quoted context omitted.

Weird. The results are in the exact same order for me when I clicked both of your links. The only difference is that the bootstrapshuffle.com result is below (or rather, on the second page) the stackoverflow result on Bing. Considering the former is specific to Bootstrap, it's less relevant than the stackoverflow result. Anyways, DDG does mainly use Bing, but they also do their own modifications to the rankings, whic…

> [...] but they also do their own modifications to the rankings, which is sometimes for better and sometimes for worse. Could you maybe share a reference for this?

I'm a couple of days late to this, but no, I cannot provide a reference (anymore). DDG used to have a page that described in pretty great detail how the results were sourced and ranked, but it seems the current page is buried pretty deep and has less information specific to ranking. IIRC, the Bing results (and previously Yahoo/Verizon Media) rankings were augmented by their own crawler (DDBot) and aggregated usage of their search. I believe they also pull the results that their "instant answers" are derived from higher up based on the search. If, for example, you did a technical search and their "instant answer" was from a stackoverflow post, that particular post would be pulled higher up the list (but not necessarily to the top) even if it was lower on Bing.

Re: Popular Pirate Sites ‘Disappear’ from DuckDuckGo’s Top Search Results

#124

Post by effie » Wed, Nov 18, 2020, 8:44 PM UTC

mschuster91 wrote: post #94

textgel wrote: post #83

Earlier quoted context omitted.

Perhaps the fixes for the problems we have shouldn't involve some patchwork bodge to the laws that align our principles but should instead start at investigating why we need these bodges in the first place? It's an unthought out idea but between existing slander laws, a reasonable expectation of common sense/reason from your average citizen and a lack of corruption in government; I'm wondering if setting up the syste…

> a reasonable expectation of common sense/reason from your average citizen Half the US has voted for someone explicitly denying science, half the UK has voted for Brexit, in some areas of Germany 25% vote for people just a few beards short of Nazis, Bolsonaro is still in office. The "average citizen" can not be trusted to vote in their own interest or in the interest of the future generations.

> The "average citizen" can not be trusted to vote in their own interest or in the interest of the future generations.

Maybe not, but in a democratic system the right to choose trumps any ideas about what you or some elitist group thinks is best for future generations.

Are you advocating for undemocratic system?

Re: Popular Pirate Sites ‘Disappear’ from DuckDuckGo’s Top Search Results

#125

Post by effie » Wed, Nov 18, 2020, 9:09 PM UTC

esarbe wrote: post #111

ghthor wrote: post #22

Earlier quoted context omitted.

It's not a conspiracy theory that people are hurt by vaccines. That's a fact. The question is, are people hurt more by the vaccine or the disease itself. From the evidence I've researched, I think it's the vaccines that are more dangerous.

That's interesting. Why don't you publish your research, so it can be peer reviewed?

In case you are serious.

https://en.wikipedia.org/wiki/1976_swine_flu_outbreak

> Three more cases of Guillain-Barré were reported in early December, and the investigation into cases of it spread to eleven states. On December 16, a one-month suspension of the vaccination program was announced by Sencer. William Foege of the CDC estimated that the incidence of Guillain-Barré was four times higher in vaccinated people than in those not receiving the swine flu vaccine.