Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Suddenly, Hacker News is not the first result for 'Hacker News'

google.com

Post reply on HN
Print view View original

201–210 of 216 posts

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#201

Post by benatkin »

billpatrianakos wrote: post #171
benatkin wrote: post #112

Earlier quoted context omitted.

I never thought that Google Search would be so laughable.

Excellent contribution... Ugh, this is why I don't mind if HN is listed on Google or not. So which part is laughable? The part where they're able to crawl the vast expanses of the web and return relevant results for the majority of their users? Or is it the part where they came out of nowhere to dominate search because they did it better than the rest? Come on now, you can't be all things to all people. Google is far…

No, the part where instead of getting better, Google search keeps getting worse. The verbatim mode is finally there but much of the time when it mucks with my search terms it's wrong. The UI is also cluttered. They are doing a lot of things that do nothing to improve the quality of the search, so when I want to learn something, I often skip Google and go straight to Stack Overflow or Wikipedia. I did jump the gun on this issue, though...turns out some googlebot blocking was involved.

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#202

Post by snprbob86 »

pilif wrote: post #188
snprbob86 wrote: post #181

Earlier quoted context omitted.

Yup. I've got real work to do. Making a validator happy is fake work.

By ensuring that your pages are valid, you make it ever so much more likely that you will not have to scramble around wasting time at a most inopportune time when the new version of a browser comes out which handles your non-standards compliant tag soup differently than the current version of the browser. So, do you want to pay the price upfront when you can plan for it or afterwards when the fix must be done immedia…

In my experience, I've had to scramble to fix browser compatibility issues every bit as often with 100% standards-compliant code, as I have had to with some incredibly laughably bad HTML.

I'd much rather pay the exact price later, than an inflated price now.

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#203

Post by jedberg »

pg wrote: post #15
Matt_Cutts wrote: post #9

I think I know what the problem is; we're detecting HN as a dead page. It's unclear whether this happened on the HN side or on Google's side, but I'm pinging the right people to ask whether we can get this fixed pretty quickly. Added: Looks like HN has been blocking Googlebot, so our automated systems started to think that HN was dead. I dropped an email to PG to ask what he'd like us to do.

I sent you an email about this. (A couple weeks ago I banned all Google crawler IPs except one. Crawlers are disproportionately bad for HN's performance because HN is optimized to serve recent stuff, which is usually in memory.)

reddit had the same problem. We set up a separate server just for the google crawler with it's own copy of the database, so that the queries for old pages didn't slow down everyone else.

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#204

Post by yeggeyeggeyegge »

pierrefar wrote: post #10

Hi I work at Google helping webmasters. It seems something has been blocking Googlebot from crawling HN, and so our algorithms think the site is dead. A very common cause is a firewall. I realize that pg has been cracking down on crawlers recently. Maybe there was an unexpected configuration change? If Googlebot is crawling too fast, you can slow it down in Webmaster Tools. I'm happy to answer any questions. This is…

Not a webmaster, but would like to know: Does/can the google crawler system send a notification email to the registered webmaster of a website, if something like this happens? This could be deemed unsolicited, but would in a huge majority of the cases be more than welcome!

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#205

Post by [deleted] »

Matt_Cutts wrote: post #34
aw3c2 wrote: post #25

Earlier quoted context omitted.

I would like to set that crawl rate but do not see why I must register at Google to do so. Why can't Google support the Crawl-Delay directive in robots.txt for this?

My plane is about to take off, but very briefly: people sometimes shoot themselves in the foot and get it way, way wrong. Like "crawl a single page from my website every five years" wrong. Crawl-Delay is (in my opinion) not the best measure. We tend to talk about "hostload," which is the inverse: the number of simultaneous connections that are allowed.

[deleted]

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#206

Post by adamsc »

Matt_Cutts wrote: post #131
AznHisoka wrote: post #108

Earlier quoted context omitted.

No, he's not a super hero.. and no he doesn't help everyone that asks for it. I've brought up many issues over the years to him via Twitter, blog comments, emails, etc, and none of them.. count them.. ZERO have been addressed by him. He just gives preferential treatment to those sites that make a lot of noise, and might give Google or Google's quality team a lot of negative press. That's all he does. I've even brough…

Hi AznHisoka. A lot of people send me spam reports, and I'm happy to pass them on. But normally we investigate the spam reports without sending back specific feedback of what action we took in response. P.S. Just curious--is this the same AznHisoka from BlackHat World, the "Blackhat SEO forum"? http://www.blackhatworld.com/blackhat-seo/members/137345-azn...

Oh noes; Matt Cuts is going to think BHW is just spammers - siiigh

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#207

Post by pierrefar »

yeggeyeggeyegge wrote: post #204
pierrefar wrote: post #10

Hi I work at Google helping webmasters. It seems something has been blocking Googlebot from crawling HN, and so our algorithms think the site is dead. A very common cause is a firewall. I realize that pg has been cracking down on crawlers recently. Maybe there was an unexpected configuration change? If Googlebot is crawling too fast, you can slow it down in Webmaster Tools. I'm happy to answer any questions. This is…

Not a webmaster, but would like to know: Does/can the google crawler system send a notification email to the registered webmaster of a website, if something like this happens? This could be deemed unsolicited, but would in a huge majority of the cases be more than welcome!

All these errors are reported in Webmaster Tools in the Diagnostics section. Verify your site and you'll have access to this and a wealth of more data.

Also, we do send notifications in Webmaster Tools, and you can confgure those to be delivered by email too. I'm not sure if we send messages for these kinds of serious crawl errors, so I'll need to check. If not, that's an interesting idea I can ask the team to think about.

Thanks for the feedback :)

Pierre

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#208

Post by bascule »

Animus7 wrote: post #186
bascule wrote: post #185

Earlier quoted context omitted.

Some of us actually care about interoperability, maintainability and writing good code in general as opposed to just cowboying stuff together as quickly as possible

I used to preach the same thing. Then, working at a startup taught me that it's not black and white. Several quotes come to mind, but Voltaire's is my favorite: "The perfect is the enemy of the good."

Having worked at startups with both cowboys with a "get 'er done" attitude and people who actually care about software craftsmanship, I'll take the latter any day.

Cowboys may get things "done" quickly, but that doesn't help when things are subtly broken, have interoperability problems, or are nearly impossible to extend without breaking.

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#209

Post by rayz0r »

Matt_Cutts wrote: post #131
AznHisoka wrote: post #108

Earlier quoted context omitted.

No, he's not a super hero.. and no he doesn't help everyone that asks for it. I've brought up many issues over the years to him via Twitter, blog comments, emails, etc, and none of them.. count them.. ZERO have been addressed by him. He just gives preferential treatment to those sites that make a lot of noise, and might give Google or Google's quality team a lot of negative press. That's all he does. I've even brough…

Hi AznHisoka. A lot of people send me spam reports, and I'm happy to pass them on. But normally we investigate the spam reports without sending back specific feedback of what action we took in response. P.S. Just curious--is this the same AznHisoka from BlackHat World, the "Blackhat SEO forum"? http://www.blackhatworld.com/blackhat-seo/members/137345-azn...

Oh snap :D

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#210

Post by billpatrianakos »

larrys wrote: post #197
billpatrianakos wrote: post #155

Earlier quoted context omitted.

I know about that and it's on purpose. There's a number of reasons I want it that way for now. Way to be snarky for no particular reason. I didn't know offering up a sincere comment about PG roused so much emotion.

Actually I wasn't being snarky. Stuff like this happens and many times the owner of the site doesn't know it since they are using a local cached copy from their browser as only one example or the dns is cached so they are the last to find out. I was just pointing it out. And the only reason I saw it was that I read your comment. (Really) I see you've now removed the domain name from your profile now.

Yeah, I did. Sorry, I took that wrong. That used to be a working domain when I first had it in the profile. I just pointed the name servers to another hosting account but I'm not quite ready to do anything with it which is why it was dead. Thanks for reminding me it was in my profile though. I did forget that.
Post reply on HN