I definitely didn't expect to see my own blog as the last result on page one. Or is that because I've shared it via Google Plus and it's being injected into my personal results?
Suddenly, Hacker News is not the first result for 'Hacker News'
151–160 of 216 posts
Re: Suddenly, Hacker News is not the first result for 'Hacker News'
#152Earlier quoted context omitted.
I believe Rtm has already set one up.
We use varnish for caching and check the useragent for requests. If the cache has a copy of an article that is a few hours old it will just give that version to Googlebot while if it thinks a human is requesting the page then it will go to the backend and fetch the latest version. https://www.varnish-cache.org/lists/pipermail/varnish-misc/2...
+1 for varnish. It's stupidly[1] fast and there shouldn't be much trickery required to deflect most of HN's traffic (e.g. ~10 sec expiry for "live" pages, infinite expiry for archived pages).
[1] 15k reqs/sec on a moderate box
Re: Suddenly, Hacker News is not the first result for 'Hacker News'
#153Still number one via DuckDuckGo. Google is now officially useless.
DDG is using Bing for most general web results.
My site mysteriously disappeared from Bing and DDG a few days ago at the same time; but Gabriel said if I'm not cool enough to get a top ranking on HN for six hours to complain about it, I'll just have to go through the normal process Bing sets up for webmasters.
Re: Suddenly, Hacker News is not the first result for 'Hacker News'
#154Earlier quoted context omitted.
Hi Paul, A site can be crawled from any number of Googlebot IP addresses, and so blocking all except one doesn't help in throttling crawling. If you verify the site in Webmaster Tools, we have a tool you can use to set a slower crawl rate for Googlebot, regardless of which specific IP address ends up crawling the site. Let me know if you need more help. Edit Detailed instructions to set a custom crawl rate: 1. Verify…
Could google vary the crawling rate on each site and see what effects that has on response times, and develop an algorithm to adjust crawl speed so as not to affect site performance too much? If google starts crawling a site and notices sequential crawl requests are answered in .5s w/ .1s stddev and it starts crawling with 10 parallel connections and the answers are 2s w/ 1s stddev, clearly that's a problem because u…
> declare how likely a page is to change in the future
I believe you can do that using a sitemap.xml
Re: Suddenly, Hacker News is not the first result for 'Hacker News'
#155Earlier quoted context omitted.
It seems he doesn't want to. From what I understand there's the possibility that he'd prefer to accomplish his goal without the use of Wemaster Tools. Like I said, he knows his shit, so he's going to do things on his terms if he can. Webmaster Tools might be what the average guy wants to use but then there are some who follow the beat of their own drummer whenever possible for whatever reasons we may never know.
Speaking of knowing ones shit your site listed in your profile billpatrianakos.com is dead - no dns being returned.
I know about that and it's on purpose. There's a number of reasons I want it that way for now. Way to be snarky for no particular reason. I didn't know offering up a sincere comment about PG roused so much emotion.
Re: Suddenly, Hacker News is not the first result for 'Hacker News'
#156Earlier quoted context omitted.
The number of webpages with HTML that's just plain wrong (and renders fine!) is staggering. I often wonder what the web would be like if web browsers threw an error upon encountering a syntax error rather than making a best effort to render. If you're writing HTML, you should be validating it: http://validator.w3.org/
Google.com has 39 errors and 2 warnings. Among other things, they don't close their body or html tags. Is there any real downside to having syntax errors?
Not really, but there are many benefits to keeping your HTML clean. Google seems to just use whatever works, in able to support all kinds of ancient browsers.
Re: Suddenly, Hacker News is not the first result for 'Hacker News'
#157Nor second or anywhere in the top five (seven it is).
Re: Suddenly, Hacker News is not the first result for 'Hacker News'
#158Does anyone else find it disturbing that google employees are bending over for pg/hn? Seriously, if any other webmaster blocked googles bots they wouldn't change their algorithms to accommodate, or see how they could use less of our resources. Its pg's fault not googles, and I dont see why they should care. Maybe from their standpoint it would be more beneficial to google users who are used to typing in 'hacker news'…
What algorithm changes?
Re: Suddenly, Hacker News is not the first result for 'Hacker News'
#159Earlier quoted context omitted.
Yes, I'm GoogleGuy. Steven Levy documented that in his recent "In The Plex" book, so there's no harm in verifying it now.
Matt, when you have a free minute, can you take a look at this? http://news.ycombinator.com/item?id=3272528
Sure, some indexing folks are discussing this now.
Re: Suddenly, Hacker News is not the first result for 'Hacker News'
#160Earlier quoted context omitted.
Matt, when you have a free minute, can you take a look at this? http://news.ycombinator.com/item?id=3272528
Sure, some indexing folks are discussing this now.
I appreciate it. I set up a 301 from dl.php?id=x to /myproduct/ for each of the ids last night as a last-ditch effort.