Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost all searches on my independent search engine are now from SEO spam bots

blog.searchmysite.net

Post reply on HN
Print view View original

181–190 of 391 posts

Re: Almost all searches on my independent search engine are now from SEO spam bots

#181

Post by EVa5I7bHFq9mnYK »

sdoering wrote: post #124
marginalia_nu wrote: post #111

Earlier quoted context omitted.

They're probably doing all they can. The problem is their dominance, both means they have effectively an entire industry looking for loopholes in everything they do, as well as legal considerations (arbitrarily punishing individual smaller actors might skirt on the territory of anti-competitive behavior)

I fear that Google also has a conflict of interest here. A lot of these non optimized sites are not interested in making money via ads. So Google wouldn't profit additionally from leading people there. And a lot of people (myself often times included) are looking for a quick answer. A good enough answer. So good enough, SEO optimized is being surfaced. The result of an optimization war on both sides combined with the…

Maybe back to Yahoo model of the 90s? Manually created collection of curated links?

Re: Almost all searches on my independent search engine are now from SEO spam bots

#182

Post by robmay »

While I'm generally a blockchain skeptic, this is actually a good use for a blockchain - to "register" bots so they have an id, and an owner, and you can measure their behavior. There are going to be more bots interacting with more sites, so, this could work.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#183

Post by Nextgrid »

ratww wrote: post #133
sdoering wrote: post #124

Earlier quoted context omitted.

I fear that Google also has a conflict of interest here. A lot of these non optimized sites are not interested in making money via ads. So Google wouldn't profit additionally from leading people there. And a lot of people (myself often times included) are looking for a quick answer. A good enough answer. So good enough, SEO optimized is being surfaced. The result of an optimization war on both sides combined with the…

I think there's two kinds of SEO spam going on. The black-hat kind is definitely made to extract money from ads. But those are easy to avoid for web veterans IMO. And I also feel that Google is doing its part, even though it's costing them money from those sweet ads! But the white-hat kind, also known as content marketing, is made to let legit companies save money. Instead of paying for Google Advertisement, they get…

The "white-hat kind" can trivially be filtered out (or deterred) by downranking any of the crap these marketers use to measure their conversion rate - analytics, etc.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#184

Post by matheusmoreira »

politician wrote: post #91
matheusmoreira wrote: post #75

Earlier quoted context omitted.

Exactly what I mean. There is no way to have an international network with national borders. Telecommunications providers have always been centralized and have always been in bed with the government. Only way we'll ever be free is if someone invents some kind of decentralized long range wireless mesh network.

Like Starlink?

Starlink is maintained by a company, it's an internet service provider. One visit from the police and they'll censor anything.

The mesh network should be made out of common hardware in order to be viable. I'd suggest phones but those devices are owned before they've even left the factory.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#185

Post by m-i-l »

fortran77 wrote: post #138
Teandw wrote: post #62

This guy throws multiple reasons/conspiracies out there on why the website is really struggling to gain literally any sort of traction. Web is all bots, search engines not promoting competitors and being drowned out by SEO spam, yet he's failing to see the most obvious reason... the reason nearly all websites don't gain traction... Because it's a bad website. It provides no value to the user. I put in a few search te…

I found a few pro-terrorism sites here. I don't think it's the OPs purpose, but he's being duped by the few users that do look for sites like this where they can add a "curated link" to their ISIS or Hezbollah or Hamas site with a slick facade.

Thanks for your feedback. If you can drop me a note I'll remove those sites - it is against the Terms of Use at https://searchmysite.net/pages/terms/ (not that spammers, terrorists, etc. care about complying with a Terms of Use). I think legitimate looking home pages as a front to other non-legitimate content is a genuine problem this model doesn't solve (also noting that some of those home pages may even be genuinely legitimate but have been hacked e.g. via a subdomain takeover).

Re: Almost all searches on my independent search engine are now from SEO spam bots

#186

Post by nyokodo »

ysavir wrote: post #136
indigochill wrote: post #72

Earlier quoted context omitted.

How do you surface organic human content? I happen to linger around the fediverse/tildeverse sphere where I see organic content from people I personally have a direct (digital) connection to (and I started self-hosting my music after Epic bought Bandcamp), but I'm not clear on how I'd go about digging that kind of stuff up in the more general case.

It's not about surfacing organic human content, it's about only indexing organic human content. The problem is automated indexing. So long as indexing works according to defined rules, the advantage will be to those able to shape their content to those rules, and the spammers and scammers will win. An idea I've had for a few years is making a social-network based index engine. The only pages that get indexed are page…

> up to 5 or 6 degrees of separation

So basically everyone on earth?

Re: Almost all searches on my independent search engine are now from SEO spam bots

#187

Post by m-i-l »

honkdaddy wrote: post #112
Teandw wrote: post #62

This guy throws multiple reasons/conspiracies out there on why the website is really struggling to gain literally any sort of traction. Web is all bots, search engines not promoting competitors and being drowned out by SEO spam, yet he's failing to see the most obvious reason... the reason nearly all websites don't gain traction... Because it's a bad website. It provides no value to the user. I put in a few search te…

Searching for Astral Codex Ten, a popular, well-written, non-spammy blog which I would expect is indexed... Returns only results in which _other_ bloggers are referencing ACX. Consider me as one of the datapoints that arrived from HN and likely won't be back, I'm afraid.

Thanks for your feedback. The idea was for people to submit sites they like, and search sites other people have liked. I've submitted Astral Codex Ten, and that site is now indexed for the benefit of others.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#188

Post by m-i-l »

native_samples wrote: post #119
Teandw wrote: post #62

This guy throws multiple reasons/conspiracies out there on why the website is really struggling to gain literally any sort of traction. Web is all bots, search engines not promoting competitors and being drowned out by SEO spam, yet he's failing to see the most obvious reason... the reason nearly all websites don't gain traction... Because it's a bad website. It provides no value to the user. I put in a few search te…

Well, it's worse than that. The whole schtick is that it's only pure, real content by folksy people like us. The top reason to use it on the about page is: Indexes only user-submitted sites with a moderation layer on top, for a community-based approach to content curation, rather than indexing the entire internet with all of its spam, "search engine optimisation" and "click-bait" content. So I tried searching [kotlin…

Thanks for your feedback. It is just the home page which is moderated before indexing (and reviewed annually). When https://personalsit.es/ was listed it looked legitimate, but agreed the results for that site look infected with spam now. I've found at least one other site today where the home page and blog look genuinely legitimate, but which has a complete spam subdomain, quite possibly the victim of a subdomain takeover attack by spammers. I've delisted both. Unfortunately it isn't an easy task trying to defeat a vast army of well funded spammers in your spare time!

Re: Almost all searches on my independent search engine are now from SEO spam bots

#189

Post by eloisius »

matheusmoreira wrote: post #75
kmlx wrote: post #65

Earlier quoted context omitted.

> I think it'll eventually fragment into multiple segregated continental networks i think it already has. the Great Firewall of China is the classic example, but I think the trend started in the west with the Right to be forgotten/right to erasure in Europe, and subsequent HTTP Status 451 Unavailable For Legal Reasons. GDPR just further cemented the split between Europe and the rest, and the new DMA & DSA regulation…

Exactly what I mean. There is no way to have an international network with national borders. Telecommunications providers have always been centralized and have always been in bed with the government. Only way we'll ever be free is if someone invents some kind of decentralized long range wireless mesh network.

Good luck, spectrum is highly regulated in every country I can think of. If national governments don’t want you networking across borders, you’re definitely not going to be broadcasting long range radio transmissions that way. In fact, it’s currently illegal to transmit encrypted data or to relay packets via ham radio in the US.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#190

Post by throw10920 »

FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

> I agree: the WWW Internet is dead

I've heard this claim a lot, with 0 supporting evidence. Do you have any?

My own experience is that there are thousands of content-rich, high-quality blogs still being written by real humans, because I regularly find and bookmark new ones weekly, without even looking for them, so: please provide evidence for this claim that runs counter to my lived experience.

Post reply on HN