Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost all searches on my independent search engine are now from SEO spam bots

blog.searchmysite.net

Post reply on HN
Print view View original

31–40 of 391 posts

Re: Almost all searches on my independent search engine are now from SEO spam bots

#31

Post by asddubs »

throwaway27727 wrote: post #25
blackhaz wrote: post #7

Earlier quoted context omitted.

Sometimes adding "reddit" to a search query produces fantastic results.

This is great but it seems reddit has done something to mess with their date reporting. When looking for recent posts, I might see a result on Google that says it was posted in the last few days, but on clicking the result will actually be from years ago.

might also be google. I've noticed inaccurate dates that don't appear anywhere for some of my pages. my only theory as to why these were displayed is that google interpreted a (server side) randomly generated number in an inline script as a timestamp (but i can't know for sure that's what happened)

Re: Almost all searches on my independent search engine are now from SEO spam bots

#32

Post by maven29 »

hihihihi1234 wrote: post #5
mensetmanusman wrote: post #4

Earlier quoted context omitted.

They face a lot. I always browse with incognito on safari, and I quite often have to do captchas on google and bing etc. to prove I’m not a computer… If there is money involved and value in being able to trick search engines, I’m not surprised it’s a thriving business of grift.

Why do you use Bing?

You should try Bing again. Bing doesn't mess with your query terms as much as google does. If you aren't a zoomer typing out whole sentences into the search bar, the fact that Bing doesn't substitute your jargon for more general terms will help with spending less time in the search results.

I just got tired of iterative refining not working as it used to in the past. I once got results for databases when searching for decibels (despite spelling it out in full), so it isn't just a matter of semantically related terms.

The rewriting is just braindead and the ranking algorithm falls for generated content way too easily. Google shouldn't be trying to teach me DHCP when I am clearly trying to recall a config item, but then it gets worse when you read the infobox and realize that it's written at a toddler level of comprehension.

This is with the caveat that all search engines rely on some level of personalization, so you might be able to get good results on google if they deem you worthy.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#33

Post by SirAiedail »

blackhaz wrote: post #7
albatrosstrophy wrote: post #6

Ona tangential note, I remember a time when Google had the option to search only for 'discussions'. The results were amazing and accurate as it scoured online forums. Almost all issue I had (was following the rooting scene closely back then) were quickly resolved. Then suddenly it got removed for reasons unknown to me. Anyone knows if it's replicatable today?

Sometimes adding "reddit" to a search query produces fantastic results.

I use "site:reddit.com" to fully restrict to that. You can even filter by subreddit that way.

Works well with HN and other sites, too.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#34

Post by Gigachad »

FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

Interesting thought. I just went though my browser history and realised that almost every time I use google search, I already know what website I want, I just don’t know the exact link/page. I’ll use google because the search on stack overflow or reddit sucks but I know I’m looking for a page on one particular site.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#35

Post by matheusmoreira »

FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

The internet itself is probably gonna die soon anyway. Every country wants to impose its own laws on it. I think it'll eventually fragment into multiple segregated continental networks, if not national ones, all with heavy filtering at the borders.

I'm happy to have experienced the free internet. Truly a jewel of humanity.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#36

Post by Hnrobert42 »

FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

As you describe this, it makes me think about how populations tend to migrate to cities and away from rural areas. There’s even a parallel to white flight in the emerging popularity of the chan/gab fora.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#37

Post by [deleted] »

mywaifuismeta wrote: post #29
FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

I no longer see Google as a neutral "search engine" the way it used to be. Now it's just another company that owns and promotes certain types of content, no different from reddit. For some things Google has the best content, for some things Twitter or Reddit have the best content.

[deleted]

Re: Almost all searches on my independent search engine are now from SEO spam bots

#38

Post by ajnin »

This made me curious to try that search engine so I typed "electronic music box" (first thing that came to mind). As far as I can tell none or the 10+ pages of results include all those 3 words. I mean, you might not have any relevant sites in your database (likely if there are only 1000 sites or so as another of your blog posts imply), and I understand you want to show some result to the user, but if I want irrelevant links I might as well go to google.com...

Re: Almost all searches on my independent search engine are now from SEO spam bots

#39

Post by oefrha »

throwaway27727 wrote: post #25
blackhaz wrote: post #7

Earlier quoted context omitted.

Sometimes adding "reddit" to a search query produces fantastic results.

This is great but it seems reddit has done something to mess with their date reporting. When looking for recent posts, I might see a result on Google that says it was posted in the last few days, but on clicking the result will actually be from years ago.

Messed up dates, plus irrelevant topics showing up because there are matched snippets in “more posts from…”.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#40

Post by cbozeman »

mywaifuismeta wrote: post #13

That's really interesting... and sad. For what it's worth, I've noticed comment bots dramatically increase over the last year too. They have always been there, but looking at Reddit, YouTube, etc, now there seem to be 10x more than there were a few years earlier. Even on HN it has gotten worse.

Is there a browser plug-in or some other piece of software that can filter, or highlight, which posts / comments are likely made by bots?
Post reply on HN