Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost all searches on my independent search engine are now from SEO spam bots

blog.searchmysite.net

Post reply on HN
Print view View original

371–380 of 391 posts

Re: Almost all searches on my independent search engine are now from SEO spam bots

#371

Post by metadat »

marginalia_nu wrote: post #335
FargaColora wrote: post #328

Earlier quoted context omitted.

These websites seem to be last updated decades ago, which is prehistoric to most casual browsers. There's no doubt there is great content on the general internet, but these examples I would classify as "legacy".

I can see why the website owners would be interested in getting traffic to recent websites, but why would you be interested in recently updated websites?

Stores typically stock recently manufactured products. Once the manufacturer discontinues a model and inventory is gone, that's a wrap. Sometimes the product was good and gets replaced by an inferior one (in the spirit of old burger king vs new post-acqusition burger king), other times it's just small tech refresh tweaks, and everything in between.

A real litter of inconsistency between unrelated external organizations and varying markets and skill sets.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#372

Post by synicalx »

blisterpeanuts wrote: post #295
john-radio wrote: post #260

Since everyone in this thread wants to jump down OP's throat about the quality of his web site, another interesting search engine is millionshort.com, which allows you to filter out the top N web sites from the results of your search. It's a great tool for looking past sites with good SEO; all you have to do is fiddle with the value of N. For example, searching for "electronic music box" as /u/ajnin suggested, with t…

That's an outstanding concept. One problem though: wouldn't it also filter out high quality curated results?

I assume the idea is this is a secondary search, after Google has failed once again to return anything other than Etsy and Pinterest results.

It also seems fairly customisable, like I can search and include all results but choose to remove ecommerce, or sites with live chat (weird filter, but I like it).

Re: Almost all searches on my independent search engine are now from SEO spam bots

#373

Post by jon_richards »

go_prodev wrote: post #351
MicahKV wrote: post #298

Earlier quoted context omitted.

I'm not so sure about this. The spammers goal is to build up as big a list of link spam targets as possible. If one spammer chooses to only scrape minor engines and another only major engines, the one scraping the major engines will probably come out on top despite the higher cost. Whoever is abusing OP's search engine is likely doing it to supplement the data they are already scraping from the major engines. For OP,…

Deliberately feeding the spam bots into an endless loop of captchas might slowly drain their accounts if they are paying 3rd party captcha farms.

Then monetize by setting up your own captcha farm, but instead of paying for compute, send the captcha to the spam bots, who send it to another captcha farm and solve it for you.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#374

Post by nl »

CWuestefeld wrote: post #236
nl wrote: post #127

Earlier quoted context omitted.

Eternal September was Sep 1993. AOL hit the internet in March 1994. Netscape didn't launch until December 1994 (and the WWW was nothing before that. I subscribed to a mailing list with new sites that were released and I'd visit most new websites on the internet on most days with the Cello browser in my uni labs most days). AOL users have been there since the beginning of the WWW. https://en.m.wikipedia.org/wiki/Etern…

My recollection is that the AOL event you reference was only making usenet accessible - a point that makes good sense in the context of the eternal September. But when talking about the WWW, that's a very different story. I think that AOL didn't incorporate a web browser until quite some time after that.

The WWW took off when Netscape shipped in late 1994.

AOL users could use Netscape from the beginning.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#375

Post by zerd »

skinnymuch wrote: post #364
kelnage wrote: post #346

Earlier quoted context omitted.

I understand your opinion about affiliate links - but I use several review websites that use such links for all products they review, and have both positive and negative reviews for products. So I wouldn’t say it necessarily follows that affiliate links = biased reviews.

How often do they give their best review score or opinion to a product without an affiliate link? Not every product will have an accessible affiliate link. Isn’t Amazon commonly used for most affiliate links or has that changed in recent years? Amazon isn’t the cheapest all the time any more. Nor is its customer support the top any more

Also, I've noticed that the list of products reviewed is limited to only those that _have_ Amazon affiliate links. If a product is only available on not-Amazon stores, they don't even get mentioned. Which is a bias in itself.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#376

Post by bornfreddy »

ysavir wrote: post #136
indigochill wrote: post #72

Earlier quoted context omitted.

How do you surface organic human content? I happen to linger around the fediverse/tildeverse sphere where I see organic content from people I personally have a direct (digital) connection to (and I started self-hosting my music after Epic bought Bandcamp), but I'm not clear on how I'd go about digging that kind of stuff up in the more general case.

It's not about surfacing organic human content, it's about only indexing organic human content. The problem is automated indexing. So long as indexing works according to defined rules, the advantage will be to those able to shape their content to those rules, and the spammers and scammers will win. An idea I've had for a few years is making a social-network based index engine. The only pages that get indexed are page…

Sounds like a great idea, execution will be key...

Re: Almost all searches on my independent search engine are now from SEO spam bots

#377

Post by ge96 »

Random tangent related to SEO I am so annoyed with random overseas companies faking local businesses by SEO

They will get your call and you can tell where you're talking to immediately. Then they get a quote that's increased to get their cut... That's then carried out by an actual local company. It's annoying because the websites at the top of the search appear local.

A specific example is when you're looking for a towing company.

Type in your state/city towing company, more than likely the top results/websites pinned to GMaps are not local-based.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#378

Post by Shinchy »

FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

I think you're thinking too narrowly about general chit chat content. E-commerce for example is still very much in the function of using your own website. As I would say is documentation, e-learning, saas, company information, etc. It's a more purposeful web.

What is dead though is the general blog like content and community platforms of old, the era of Wordpress blogs, forums and hobbyist websites is certainly gone.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#379

Post by Schroedingersat »

miohtama wrote: post #356
marginalia_nu wrote: post #350

Earlier quoted context omitted.

> So spammers have latched onto your search engine because they are getting useful results. I'm not sure about this. At least with my search engine, it doesn't really seem to matter what response they get, I don't even think they look at the responses. They keep hammering away with tens of thousands of queries per day with the requests even though they've seen nothing but HTTP Status 403 since last October or so. My…

Put a CloudFlare web application firewall at the front of the site and then use its rate limited / CAPTCHA features to throttle traffic. It is the easiest way to get rid of parasitic scraping and API abuse. Cost is $0.

> Cost is $0.

Cost is the slow enclosure of the internet hy a handful of giant companies and once attestation is universal having anyone without a locked down device be locked out of most of the internet without providing endless free labour.

FTFY

Re: Almost all searches on my independent search engine are now from SEO spam bots

#380

Post by Schroedingersat »

FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

If they wanted good content, they shouldn't have coerced everyone with good content into turning it into illegible seo spam in order to appear in the top 10 pages. The writing was on the wall the first time a recipe site had to start writing stupid stories about their dog.
Post reply on HN