Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost all searches on my independent search engine are now from SEO spam bots

blog.searchmysite.net

Post reply on HN
Print view View original

311–320 of 391 posts

Re: Almost all searches on my independent search engine are now from SEO spam bots

#311

Post by Ikatza »

z3t4 wrote: post #271
MicahKV wrote: post #252

So spammers have latched onto your search engine because they are getting useful results. They are able to systematically discover websites built on certain platforms that allow users to post content containing links, which they can target for link spam. It is very difficult to fight this on a technical level because there is an entire industry built around blackhat SEO, with all kinds of softwares and services dedic…

Or put those query results behind an anti-bot/"capcha" test.

How about serving bots with one link per page, and taking a minute to serve each page? Would this impact their efficiency?

Re: Almost all searches on my independent search engine are now from SEO spam bots

#312

Post by mdoms »

john-radio wrote: post #260

Since everyone in this thread wants to jump down OP's throat about the quality of his web site, another interesting search engine is millionshort.com, which allows you to filter out the top N web sites from the results of your search. It's a great tool for looking past sites with good SEO; all you have to do is fiddle with the value of N. For example, searching for "electronic music box" as /u/ajnin suggested, with t…

Million Short also has an option to remove only e-commerce results which is invaluable if you still want results from sites like Twitter, Wikipedia and YouTube but don't want online shopping spam.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#313

Post by big_blind »

dixego wrote: post #242
mywaifuismeta wrote: post #29

Earlier quoted context omitted.

I no longer see Google as a neutral "search engine" the way it used to be. Now it's just another company that owns and promotes certain types of content, no different from reddit. For some things Google has the best content, for some things Twitter or Reddit have the best content.

Google is an advertising company. It has been for a good while.

Yeah I use you.com and kagi.com. No advertising on either. Less SEO spam too it seems.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#314

Post by big_blind »

rchaud wrote: post #174
FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

> Want to become rich? Make a search engine which indexes the fresh relevant data from the big siloed websites, and ignores the general dead Internet. That would be a great service, but it certainly wouldn't make you rich. Where's the money going to come from? Google got rich because they acquired an ads platform (DoubleClick) and an analytics platform (Urchin) and started monetizing the vast amounts of data they had…

I use beta search engines. On kagi.com and you.com you can preference and filter top sites. There's also no advertising on either. I've just stopped using Google altogether and its improved search so much.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#316

Post by big_blind »

ouid wrote: post #235
FargaColora wrote: post #27

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just…

Google is still pretty good at searching reddit. Maybe reddit can acquire them.

site:reddit just is the best search engine at this point. I still don't like Google though.

Re: Almost all searches on my independent search engine are now from SEO spam bots

#320

Post by getcrunk »

alphabet9000 wrote: post #315

i recently built a habitat for spam bots, they eventually found it and now post peacefully https://upstairs.treehouse.telnet.asia/pharm/cylohexapine

this is the best thing I have ever seen. Its art, engineering, biology and sociology. Do you write blog posts about it?
Post reply on HN