That robots.txt is so big that even with comments maybe google will drop it after trying to parse it for a while....
Robots.txt
21–30 of 142 posts
Re: Robots.txt
#22I always thought SEO was supposed to be a cat/mouse game. The search engine tries to balance the search results fairly. The SEO consultant tries to game the system and get certain links to appear ranked higher, whether fair or not. Isn't that just how this works?
Re: Robots.txt
#23Am i the only one who thinks that SEO is more a plague then it is a real profession? Google making it hard for SEO is exactly the right thing to do. Making it impossible for SEO should be the ultimate _goal_ of a good search engine.
I think this is a common view, and I share it: I really don't like SEO. The thing is, SEO does provide real value, but it provides value because the system is broken: your website should reach the top of search results based on merit, not based on some arcane black magic.
I think it's somewhat akin to lobbyists: I don't blame them for making money off and taking advantage off the political system, and they certainly provide something of value to their customers, but I dislike the fact that they can exist at all. But that's not the lobbyist's fault, it's the system. In this respect, I applaud Google for keeping people on their toes and making SEO less relevant. That's their job after all.
Re: Robots.txt
#24I thought he was gonna ask for a job.
me too. in that case it would have been cool.
Re: Robots.txt
#25I think SEO is practically dead, and for good. The Social Web is supposedly lot more democratic and apparently a better choice.
I think any good social marketers (as they've now rebranded themselves) are fully involved in subverting The Social Web. Any site that reaches a certain size (Digg, Reddit, etc.) will become a ripe target for less than organic posts.
Re: Robots.txt
#26An SEO consultant complaining about Google's frequent algo changes is like a mechanic complaining that cars break down too often.
It's more like new cars being introduced without service manuals, forcing the mechanic to continually figure out how to fix them.
I don't think this is a good analogy. Google's job is to make sure good content is ranked high and crap is ranked low. You should just create content that is useful to others and let Google figure out how to discover it and rank it high (they are good at it).
To create useful content, you don't need to know Google internals.
Re: Robots.txt
#27Keep up the good work Google. I hope this serves as encouragement for them to continue to screw over those who want to turn searching for relevant information into a war for who can cheat the system the best.
Exactly what I thought, SEO should not have ever become a "thing" if Google did its job right.
Re: Robots.txt
#28Re: Robots.txt
#29That robots.txt is so big that even with comments maybe google will drop it after trying to parse it for a while....
I would hope that google, of all companies, would be capable of writing a crawler that ignores robots.txt comments.
Other than that the guy only has one rule.
Re: Robots.txt
#30I wonder how this post got to the top even though the sentiment in the comments is that this posts sucks.
I find the perspective of the SEO guy interesting, even if I find what he does kind of skeezy and annoying.