This is for comment spam. It's trying to find a long tail of popular but not top listed blogs for the purpose of posting comments with the much desired links to the SEO target.
Almost all searches on my independent search engine are now from SEO spam bots
41–50 of 391 posts
Re: Almost all searches on my independent search engine are now from SEO spam bots
#42This made me curious to try that search engine so I typed "electronic music box" (first thing that came to mind). As far as I can tell none or the 10+ pages of results include all those 3 words. I mean, you might not have any relevant sites in your database (likely if there are only 1000 sites or so as another of your blog posts imply), and I understand you want to show some result to the user, but if I want irreleva…
Re: Almost all searches on my independent search engine are now from SEO spam bots
#43You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…
The real dark-net is facebook. Everything that goes in there never comes out again and is basically invisible to the world, except if you join facebook yourself.
My own prime example of that used to be pinterest: it seems to be a 100% sink in the directed graph of internet links. But since Applebaum stated this, instagram (also facebook of course) is trying hard to push pinterest off that particular throne.
Re: Almost all searches on my independent search engine are now from SEO spam bots
#44One day we'll have an internet for humans exclusively. On another note, with 160K requests / day from bots you could of course simply block the bots structurally assuming they are nice enough to identify themselves. Block all of AWS and Google, Russia, China, NK and a couple of other bot hot spots and the service may well become more successful for regular users because they get faster results. Bots can afford to wai…
I wonder how that could be accomplished. Maybe they'll build a brain interface to replace the "I'm not a robot" captchas/add a TPM chip to the brain. And then the spammers will start selling tools to fake the responses. Or pay Filipinos a few cents a month to have the chip implanted to their brains...
That's the problem with blocking bots as opposed to malicious behavior. Bot blocking is actually trivial and very cheap to bypass as long as you can buy slave-like labor for peanuts.
Ideally you'd want to block malicious behavior (when it comes to SEO spam, downrank anything for-profit such as ads, analytics, affiliate links, etc) instead to remove the incentives for spamming, regardless of whether it's a bot or human.
In this case the only problem is that this search engine gives away resources (search queries) for free and then complains that people (in this case spammers) are taking it. It's not really a spam problem - they'd complain equally well if they had some legitimate user that happened to need tons of search queries to achieve their task.
The only solution here is to start charging for stuff that costs money, and then it doesn't really matter who is on the other side, as long as they pay the bill.
Re: Almost all searches on my independent search engine are now from SEO spam bots
#45You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…
Re: Almost all searches on my independent search engine are now from SEO spam bots
#46Ona tangential note, I remember a time when Google had the option to search only for 'discussions'. The results were amazing and accurate as it scoured online forums. Almost all issue I had (was following the rooting scene closely back then) were quickly resolved. Then suddenly it got removed for reasons unknown to me. Anyone knows if it's replicatable today?
Sometimes adding "reddit" to a search query produces fantastic results.
God I hope they never find out about this site.
Re: Almost all searches on my independent search engine are now from SEO spam bots
#47You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…
These discussions always make me recal Jacob Applebaum. Think of him what you want, but this statement of his really stuck with me at the time. Paraphrasing: The real dark-net is facebook. Everything that goes in there never comes out again and is basically invisible to the world, except if you join facebook yourself. My own prime example of that used to be pinterest: it seems to be a 100% sink in the directed graph…
Re: Almost all searches on my independent search engine are now from SEO spam bots
#48You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…
Re: Almost all searches on my independent search engine are now from SEO spam bots
#49Ona tangential note, I remember a time when Google had the option to search only for 'discussions'. The results were amazing and accurate as it scoured online forums. Almost all issue I had (was following the rooting scene closely back then) were quickly resolved. Then suddenly it got removed for reasons unknown to me. Anyone knows if it's replicatable today?
There's even pretty convincing looking accounts and messages that turn out to be spam in the end, once they start trying to post links.
I have Akismet on the comment section of the Wordpress front-end of the site I run, it basically said something like 99.99% of attempted comments were spam. I'm sure the same applies to e-mail and the like.
Re: Almost all searches on my independent search engine are now from SEO spam bots
#50You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…