Live data from Hacker News

Almost all searches on my independent search engine are now from SEO spam bots

blog.searchmysite.net

1–10 of 391 posts

Re: Almost all searches on my independent search engine are now from SEO spam bots

#4

If this was the spam for a search engine (almost) nobody uses, it makes you wonder how much abuse the major search engines face

They face a lot. I always browse with incognito on safari, and I quite often have to do captchas on google and bing etc. to prove I’m not a computer…

If there is money involved and value in being able to trick search engines, I’m not surprised it’s a thriving business of grift.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#5

If this was the spam for a search engine (almost) nobody uses, it makes you wonder how much abuse the major search engines face

They face a lot. I always browse with incognito on safari, and I quite often have to do captchas on google and bing etc. to prove I’m not a computer… If there is money involved and value in being able to trick search engines, I’m not surprised it’s a thriving business of grift.

Why do you use Bing?

Re: Almost all searches on my independent search engine are now from SEO spam bots

#6
Ona tangential note, I remember a time when Google had the option to search only for 'discussions'. The results were amazing and accurate as it scoured online forums. Almost all issue I had (was following the rooting scene closely back then) were quickly resolved. Then suddenly it got removed for reasons unknown to me. Anyone knows if it's replicatable today?

Re: Almost all searches on my independent search engine are now from SEO spam bots

#7

Ona tangential note, I remember a time when Google had the option to search only for 'discussions'. The results were amazing and accurate as it scoured online forums. Almost all issue I had (was following the rooting scene closely back then) were quickly resolved. Then suddenly it got removed for reasons unknown to me. Anyone knows if it's replicatable today?

Sometimes adding "reddit" to a search query produces fantastic results.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#9
One day we'll have an internet for humans exclusively. On another note, with 160K requests / day from bots you could of course simply block the bots structurally assuming they are nice enough to identify themselves. Block all of AWS and Google, Russia, China, NK and a couple of other bot hot spots and the service may well become more successful for regular users because they get faster results. Bots can afford to wait, humans are often impatient. And with 2 hits / second by bots that may well become a factor.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#10
> I didn’t notice at first because the web analytics only shows real users, and the unusual activity could only be seen by looking at the server logs.

Sounds like everyone blocking analytics (Plausible in this case), e.g. myself just now, is lumped in with spam bots.

Of course, analytics blocking can’t meaningfully swing the ~99.99% statistic.

Post reply on HN