Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost all searches on my independent search engine are now from SEO spam bots

blog.searchmysite.net

Post reply on HN
Print view View original

141–150 of 391 posts

Re: Almost all searches on my independent search engine are now from SEO spam bots

#141

Post by pmontra »

marginalia_nu wrote: post #54
FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

I built my search engine in part to explore whether this was actually true, and I don't think it actually is. There's still a lot of organic human-made content still out there, possibly more than ever, it's just not able to compete with the SEO industry that completely displaces it from Google and social media.

I take myself as an example.

People that know me and don't meet me regularly might know the URL of my web site and might care to look at it once per year and check if there is something new. Usually pictures and tales from holidays. Covid made those holidays less memorable so I didn't make any update since fall 2019. People that meet me regularly don't need that website, I'm telling them the tales first hand and showing them the pictures without being obnoxious. I guess that this website is a target for your search engine except it's not in English and your search engine seems to want English search phrases.

I don't have anything of value to share on a public chat like Twitter and I don't have an ego to pretend I do. I also don't use Facebook anymore. I go there once per year to like the messages that wish me happy birthday. I think it's polite to do so. All my media production is on WhatsApp or Telegram in group chats with people I know in real life.

If I really cared about producing content for the world I'd probably be using Twitter, Medium or the fad of the year and they'd take care of my SEO (do they?) or I'd be trying to score points on StackOverflow.

To recap: I never intended to compete on SEO. I'm really OK that my website is only for friends and spreads by word of mouth. It probably never did, I bet it's been on a flatline since I created it 20+ years ago.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#142

Post by stackbutterflow »

FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

I think you're generalizing your own behavior. I regularly use google to search for topics that cross my mind and I end up on many websites that are not one the giants in your list. It's a fun activity. If people stick to the same 10 websites that's on them. Nothing prevents you from exploring the web.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#143

Post by stuff4ben »

Just posting here that I'm real and I'm glad I found searchmysite! After HN, Verge, Ars, Gizmodo, and some car forum, I struggle to find content I want to read. Hopefully this will allow me to continue to find something I can read as I work on solving problems at work. I find distractions help me to refocus in an odd way.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#145

Post by mrtksn »

FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

It has been dead for a while now and the whole society feels it globally. Things were getting so good then things become horrible and whoever cracks the path to the goods stuff again will find great riches at the end of the path.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#146

Post by ElevenLathe »

pnutjam wrote: post #115
Cthulhu_ wrote: post #45

Earlier quoted context omitted.

I don't believe the WWW internet is dead; there's still millions of webpages being made and published every day. However, the traffic numbers are skewed in favor of the big socials and aggregators; I wouldn't be surprised if the 80/20 rule applies there.

There seems to be a tendancy towards video that undercuts the "old internet". I prefer instructions in a text or list format, but that's almost impossible to find for things like, changing the headlight bulb on my traverse. 1. turn the wheel so it is pointed hard in the direction of the bulb you are changing. 2. remove the hex screws from the shroud in the wheel well 3. pull the shroud down, it's pretty flexible plas…

I think this is actually because real, fluent literacy is still rare even in highly developed places. It may be easier for a very literate someone to dash off those instructions but most people are 1000x more comfortable making a little video. Same goes for reading vs watching the video.

This is my same theory about meetings being universally preferred to asynchronous email, even when literally all the questions someone asks at a meeting have already been answered in my long form email.

Most people, even if they can read, are not really comfortable with it. Doubly so for writing. There used to be no choice to function in society, but increasingly we can use technology to substitute for reading and writing effectively, so people do.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#147

Post by phkahler »

>> This time I’m really not sure what the solution is.

As with everything internet, the solution is to have solid, verifiable user identification. I realize the downside is that sites would love to have all your activity logged under a verifiable identity, so the other problem is we need to ban collection of such personal data.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#148

Post by galangalalgol »

sdoering wrote: post #124
marginalia_nu wrote: post #111

Earlier quoted context omitted.

They're probably doing all they can. The problem is their dominance, both means they have effectively an entire industry looking for loopholes in everything they do, as well as legal considerations (arbitrarily punishing individual smaller actors might skirt on the territory of anti-competitive behavior)

I fear that Google also has a conflict of interest here. A lot of these non optimized sites are not interested in making money via ads. So Google wouldn't profit additionally from leading people there. And a lot of people (myself often times included) are looking for a quick answer. A good enough answer. So good enough, SEO optimized is being surfaced. The result of an optimization war on both sides combined with the…

Does anyone have an ad free search engine? You'd start with blacklists from ublock origin, pi-hole, and similar, don't bother even crawling those, then have easy reporting for new or self hosted ads. Not much money in it if any, but it would be refreshing. Might even have a mode to nix anything with a payment method on the site, or that links to a site with a payment method.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#149

Post by dotnet00 »

FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

I agree that this seems way too reductive. I was recently reflecting on this and noticed that I constantly run across new blogs and sites whenever trying to learn something. I just don't usually pay much attention to the site name in the way that I remember HN, Reddit, Twitter etc.

So, while I would agree that some aspects of the old internet are dead (like 'small' ~1000 user forums focused on specific topics having largely been replaced by generally inferior subreddits and discord servers), I think it hasn't gotten as bad as you're making it out to be.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#150

Post by pwdisswordfish9 »

api wrote: post #70
marginalia_nu wrote: post #54

Earlier quoted context omitted.

I built my search engine in part to explore whether this was actually true, and I don't think it actually is. There's still a lot of organic human-made content still out there, possibly more than ever, it's just not able to compete with the SEO industry that completely displaces it from Google and social media.

All open systems are destroyed by spam once they become popular enough to be profitable targets. This will eventually happen to the Fediverse too. If there is money to be made pissing all over the commons, the commons will be pissed all over. It even happens to proprietary silos if they are too open. Look at how many bots and spammers infest social media. Propaganda and disinformation can also be considered a form of…

> This will eventually happen to the Fediverse too.

Oh, don’t worry, the Fediverse will never catch on.

Post reply on HN