Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Filters to block and remove copycat-websites from DuckDuckGo, Google and other

github.com

Post reply on HN
Print view View original

71–80 of 114 posts

Re: Filters to block and remove copycat-websites from DuckDuckGo, Google and other

#71

Post by hda2 »

Two important questions I don't see answered in the readme:

1. What is the criteria for a github copycat?

2. What is the process for having a website removed?

I ask #1 because many businesses use their own git-hosting solutions to host their code but also use github as a mirror. It would be very easy for a competitor to get the website of a rival business listed if the criteria is not strict and specific enough. I recommend never blocking the entire domain unless the website is a repeat offender (to avoid mistakenly harming innocent businesses and the liability that it may cause).

I ask #2 because many of these domains will likely be registered by honest people once the scammers are finished with them. There needs to be a way for the new owners to get their new domains delisted.

Re: Filters to block and remove copycat-websites from DuckDuckGo, Google and other

#72

Post by nerdbaggy »

kevin_thibedeau wrote: post #70

I'm seeing a lot of Github clones on DDG lately. Some even stealing the GH favicon. I can't fathom what their goal is other than to inject exploits into devs who are high value targets. You'd think a responsible search engine would be culling these for the sake of general security.

I’ve been seeing that a ton on Google as well. It seems to be the GitHub topics

Re: Filters to block and remove copycat-websites from DuckDuckGo, Google and other

#73

Post by wyclif »

conanite wrote: post #65
richardfey wrote: post #54

Earlier quoted context omitted.

Pinterest is one of those sites that I would block forever, if I could customise my search experience.

Add this to your browser's search engine list and make it default: {google:baseURL}search?q=%s+-site:pinterest.*

If on Firefox, do you need to add the optional search bar (with the magnifying glass) in order to access "Change Search Settings"? Because I'm not finding it anywhere.

Re: Filters to block and remove copycat-websites from DuckDuckGo, Google and other

#74

Post by fennecs »

Gigachad wrote: post #69
blacksmith_tb wrote: post #46

Earlier quoted context omitted.

So are you suggesting that GOOG is using bounce rate on the target site (assuming it uses gAnalytics I take it - otherwise how would they know?) to alter the ranking of search results (like "90% of users bounce in the first two seconds on scam.copycat.com/seemed-useful so we're demoting its rank by 20%"?) That would be interesting, I was musing more about if GOOG does any session tracking on their end to try and infe…

Stop using stock ticker names to refer to companies. It's cringe and not even easier to type.

I agree, but even more generally avoid abbreviations. A reader has to spend time thinking about what you are talking about. It’s detracting.

Re: Filters to block and remove copycat-websites from DuckDuckGo, Google and other

#75

Post by aniforprez »

hda2 wrote: post #71

Two important questions I don't see answered in the readme: 1. What is the criteria for a github copycat? 2. What is the process for having a website removed? I ask #1 because many businesses use their own git-hosting solutions to host their code but also use github as a mirror. It would be very easy for a competitor to get the website of a rival business listed if the criteria is not strict and specific enough. I re…

For #2 I assume a PR to remove the site could be sent and merged

For #1 it seems to mostly be a list of SEO gaming sites that I've personally found to be supremely irritating and deserve to be in the list. They basically just mirror stack overflow and GitHub issues and provide obscured links back to the original source to make sure people stay on their site. You can peruse the list in the code yourself. It's just a text file

Re: Filters to block and remove copycat-websites from DuckDuckGo, Google and other

#76

Post by Buttons840 »

Gigachad wrote: post #68
xahrepap wrote: post #59

I know there’s some serious chicken/egg issue here, and solves a different problem than Google or DDG. but I think what I need is a way to “Like” or “Favorite” a page I’m on (maybe just a bookmark?) and that triggers my personal search indexer which will then index that page. Then maybe optionally “follow” other people’s favorites as part of your own results. Imagine someone crawling Twitter or SoF and you just “foll…

In the end there is no perfect solution because the whole thing is gamed too much. Eventually spam sites will start paying popular users to like their spam page.

This attitude is too defeatist, and I'm not convinced. The OP's suggestion was that people would choose which other "favorite lists" or "indexes" they follow, this implies these are public. Who's going to follow someone who puts a StackOverflow clone filled with ads on their list? And if they do, and it shows in my results, I should be able to discover that someone I'm "following" has been compromised, and I'll unfollow them.

Re: Filters to block and remove copycat-websites from DuckDuckGo, Google and other

#77

Post by zhte415 »

slig wrote: post #32
mcfedr wrote: post #21

I never understood why Google isn't blocking these crap results, it's really making my experience of search really bad for a light of my searches

They used to be superb on detecting duplicated content. They also were extremely good at detecting spam/ham. Nowadays it feels like they don't even care anymore and whatever filters they have are either broken or untrained.

Copycat sites also used to be extremely careful at not appearing copycat sites. Or not duplicating content on the same site. I am surely not alone in recalling the old mantra of not duplicating content.

Copycat sites don't seem to care anymore.

I don't believe there are an overwhelming number for Google et al to deal with as it's often the same names topping search results that such filters can remove through semi-manual user action.

While leads to the conclusion - Google don't care about duplicate content any more.

Re: Filters to block and remove copycat-websites from DuckDuckGo, Google and other

#79

Post by D-Coder »

worik wrote: post #62
brimble wrote: post #27

Earlier quoted context omitted.

Do they do a good job at getting clickthroughs on Google ads on their site? :-/ Does the rate of ad-clicking on the results page increase if most of the "natural" results are crap? :-(

Don't be evil

I think they shut down that app.

Re: Filters to block and remove copycat-websites from DuckDuckGo, Google and other

#80

Post by LordDragonfang »

conanite wrote: post #65
richardfey wrote: post #54

Earlier quoted context omitted.

Pinterest is one of those sites that I would block forever, if I could customise my search experience.

Add this to your browser's search engine list and make it default: {google:baseURL}search?q=%s+-site:pinterest.*

Doesn't help with reverse image searches, which accounts for probably 90% of the times I end up clicking on a pinterest link.
Post reply on HN