Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Suddenly, Hacker News is not the first result for 'Hacker News'

google.com

Post reply on HN
Print view View original

91–100 of 216 posts

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#91

Post by wanorris »

storborg wrote: post #72
arctangent wrote: post #28

Earlier quoted context omitted.

Do you support/respect the "crawl-delay" directive? http://en.wikipedia.org/wiki/Robots_exclusion_standard#Crawl...

Nope. As mentioned above, apparently Google thinks people will "shoot themselves in the foot" with the crawl-delay directive, while they won't with Google's special interface (which requires registering and logging in).

I can't imagine that they are just guessing abut this. I'm sure someone tried implementing it and was horrified at the actual results before they gave up on it.

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#92

Post by [deleted] »

Matt_Cutts wrote: post #24
markerdmann wrote: post #11

Has Google been making significant changes to the search ranking algorithms in the past couple months? I've noticed a significant decline in the quality of results, to the point that (for the first time in years), I've bounced over to DuckDuckGo or Bing to try my luck there. I love Google as a company, so (if this isn't just in my head) I'd love to see things get better again. EDIT: Looks like the change to HN's rank…

If you can look back in your search history to find the specific searches that didn't do well for you, we're always happy to get really concrete examples. The best reports look like "I did a search for [Bavarian red widgets] and the results weren't good because e.g. you were missing a specific page X that you used to return or should return, or you returned Austrian red widgets" or whatever. Lots of Googlers are clea…

[deleted]

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#93

Post by billpatrianakos »

wmf wrote: post #89
billpatrianakos wrote: post #74

Earlier quoted context omitted.

I see your point and maybe my attitude toward it is totally biased or morally/ethically off balance but I think it's just fine. PG knows his shit. He knows how to block crawlers, he knows how get SEO done, he's no dummy. "So what?" you say. Well I work in web development and I meet with people all the time and explain to them how to run a site and get rankings up, etc. I tell them I'll give them the tools, show them…

PG knows his shit. He knows how to block crawlers, he knows how get SEO done, he's no dummy. Apparently he doesn't use Google Webmaster Tools, though.

It seems he doesn't want to. From what I understand there's the possibility that he'd prefer to accomplish his goal without the use of Wemaster Tools. Like I said, he knows his shit, so he's going to do things on his terms if he can. Webmaster Tools might be what the average guy wants to use but then there are some who follow the beat of their own drummer whenever possible for whatever reasons we may never know.

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#94

Post by rumpelstiltskin »

Matt_Cutts wrote: post #24
markerdmann wrote: post #11

Has Google been making significant changes to the search ranking algorithms in the past couple months? I've noticed a significant decline in the quality of results, to the point that (for the first time in years), I've bounced over to DuckDuckGo or Bing to try my luck there. I love Google as a company, so (if this isn't just in my head) I'd love to see things get better again. EDIT: Looks like the change to HN's rank…

If you can look back in your search history to find the specific searches that didn't do well for you, we're always happy to get really concrete examples. The best reports look like "I did a search for [Bavarian red widgets] and the results weren't good because e.g. you were missing a specific page X that you used to return or should return, or you returned Austrian red widgets" or whatever. Lots of Googlers are clea…

Here's a search that tripped me up -

I'm in Houston, TX. When I searched for 'windshield repair houston', the 1st result - wwwDOThoustonwindshieldrepairDOTnet - looked promising so I made an appointment with them to get my windshield repaired. When I went to their place of business, it was just a guy in a pickup truck in the parking lot of a strip mall, with a 'windshield repair' sign on the back of the truck.

Turns out he's running a scam where he gets ppl to file claims with their insurance, and when they pay him, he would kickback 50% to the customer. Having insurance pay for damages is common enough, plenty of businesses do it. But this guy was trying to get me to file claims for damages that I didn't even have. When I asked for a cash price just to repair the damage I did have, he refused saying that 'it wasn't enough money'.

I was pissed. Not only is this illegal, I couldn't believe he was ranked #1. When I did some digging, it turns out the guy is gaming google with a ton of paid backlinks. For example, http://www.searchpicks.com/business/automotive/patsco-windsh... (click 'suggest listing' for the price).

I'm sure plenty of other searchers ended up wasting their time with this SERP just like I did.

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#95

Post by huxley »

officemonkey wrote: post #85
Matt_Cutts wrote: post #34

Earlier quoted context omitted.

My plane is about to take off, but very briefly: people sometimes shoot themselves in the foot and get it way, way wrong. Like "crawl a single page from my website every five years" wrong. Crawl-Delay is (in my opinion) not the best measure. We tend to talk about "hostload," which is the inverse: the number of simultaneous connections that are allowed.

I would think that the number of people who (a) know how to create a valid robot.txt file, (b) have some idea of how to use the "crawl-delay" directive and (c) write a "shoot-themselves-in-the-foot" worthy error is vanishingly small.

As opposed to --for illustrative purposes-- the vanishingly small number of people who know how to block IP addresses and manage to get their site to disappear from Google's listings?

::cough::

A few years ago, I did pretty much the same thing myself. Thankfully the late summer was our slow season and the site recovered pretty quickly from my bone-headed move, but the split second after I realized what I've done was bone-chilling.

I think just about everyone has thought at some point that they understood how something worked, only to have had things go pear-shaped on them.

The lesson: people are not fully knowledgeable about everything, even the smart and talented ones.

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#97

Post by zem »

hahla wrote: post #29

Does anyone else find it disturbing that google employees are bending over for pg/hn? Seriously, if any other webmaster blocked googles bots they wouldn't change their algorithms to accommodate, or see how they could use less of our resources. Its pg's fault not googles, and I dont see why they should care. Maybe from their standpoint it would be more beneficial to google users who are used to typing in 'hacker news'…

> Maybe from their standpoint it would be more beneficial to google users who are used to typing in 'hacker news' to visit this site, but since when did that matter to google?

from google's point of view, a user who types "hacker news" into google is almost certainly looking for news.ycombinator. therefore, it is highly desirable for the site to be the #1 result.

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#98

Post by zem »

orijing wrote: post #87
Tichy wrote: post #70

Earlier quoted context omitted.

Google should simply deliver the best results, no matter what algorithm they use to do so.

If that were truly their goal, their ads would never be the useful, because if they were the best results, they would be duplicates of the organic search results (hence useless). Or, if they weren't the best results, then the best results are in the organic search. Clearly that's not the case, as advertising makes 99% of their revenues.

the point of ads is not to be maximally useful, it's to maximise some function of (usefulness to you, profitability to the advertiser).

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#99

Post by larrys »

blhack wrote: post #47
Matt_Cutts wrote: post #9

I think I know what the problem is; we're detecting HN as a dead page. It's unclear whether this happened on the HN side or on Google's side, but I'm pinging the right people to ask whether we can get this fixed pretty quickly. Added: Looks like HN has been blocking Googlebot, so our automated systems started to think that HN was dead. I dropped an email to PG to ask what he'd like us to do.

Seeing threads like this remind me that HN is still a pretty tight-knit community of real people doing real things. It's good to see this stuff sometimes. Thanks, Matt! edit: And then reading some of the other threads on this topic is a bit...something. Guys, can you calm the conspiracy theory nonsense a bit? Please? If you're not on this site very much, you might not realize that Matt pops into almost every thread w…

"This isn't HN getting some sort of preferential treatment, this is just the effect of having a userbase full of hackers"

Of course it's preferential treatment. And if you scan the last month or two of Matt's comments they are general in nature and not specific as in:

"I think I know what the problem is; we're detecting HN as a dead page. It's unclear whether this happened on the HN side or on Google's side, but I'm pinging the right people to ask whether we can get this fixed pretty quickly."

You don't think "pinging the right people" and "get this fixed pretty quickly" is preferential treatment?

Re: Suddenly, Hacker News is not the first result for 'Hacker News'

#100

Post by jaylevitt »

felipe wrote: post #58
unreal37 wrote: post #33

Earlier quoted context omitted.

It's not disturbing at all. I am glad Google is a company made up of humans and not uncaring robots. Human nature is that people want to help others when they are in a position to do so. I would do the same if HN needed my help. Of course, there are people who want to take advantage of people like Matt to give themselves an economic advantage, and I have no sympathy for those people if they can't get someone in Googl…

> I am glad Google is a company made up of humans and not uncaring robots That's exactly the problem. Google is notorious for providing horrible customer support, so why some people get personalized help, while the rest of us are stuck with the uncaring robots ?

Actually, much as I've blasted[1] Google over the years for not doing anything (like abuse-desk staffing) that requires People Not Algorithms...

Giving occasional personalized help is a great way to avoid a total black hole while still creating an escalation path. It's like having EXPLAIN QUERY stack traces on NewRelic. You don't need a deep trace of every transaction; you need a deep trace from a few representative squeaky wheels, and you use that information to improve the underlying self-serve process. This is something Google, with their "1/3 of the company parses logs", excels at.

You'll note neither Matt nor Pierre offered to do anything special for PG. They didn't say "Oh! I love HN, and we want it to succeed; I'll go push out a manual override that makes sure we use only the one crawler you unblocked." They gave him exactly the info he'd get from Webmaster Tools plus general knowledge, and told him how to use Webmaster Tools to fix it. Maybe they'll go figure out how to implement that "you're blocked!" hover-over in SERPs, but again, that would benefit everyone, not just HN. There's no special treatment here.

[1] http://blog.wordtothewise.com/2008/04/troubleshooting-a-post...

Post reply on HN