Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How rel=nofollow Works

luigimontanez.com

Post reply on HN
Print view View original

41–50 of 52 posts

Re: How rel=nofollow Works

#41

Post by mapgrep »

dolbz wrote: post #10
chalst wrote: post #5

Earlier quoted context omitted.

Google has said that it won't add a URL to its index of URLs to crawl as a result of seeing the URL in a rel=nofollow href [1]. This is over and above not weighting the link if the URL does otherwise make it onto its to crawl list. [1] http://googleblog.blogspot.com/2007/02/robots-exclusion-prot...

The post you're linking is discussing the META tag version of nofollow, not the hyperlink rel attribute which is detailed here: http://googleblog.blogspot.com/2005/01/preventing-comment-sp... This article doesn't say explicitly that it won't follow those links and I would suggest it often will as comments are a rich source of pages to index. The nofollow on a link like this just indicates that the page owner can't vo…

"who are you to tell Google they can't index a third party domain?"

No tweets link to third-party domains, since Twitter wraps all links in their own URL shortener (t.co). Even if you use an independent URL shortener, it will be wrapped by t.co.

(I suppose you could argue that there is no way for Google to determine this algorithmically, since "twitter.com" != "t.co", so it should go ahead with the crawl, but there's the question of how Twitter would respond to that.)

Re: How rel=nofollow Works

#42

Post by nknight »

abalone wrote: post #39
chalst wrote: post #3

The cited Danny Sullivan article has a little of his interview with Eric Schmidt: sullivan> I countered that Google seemed to have all the permission it needed, in that they’re not blocked from crawling pages. schmidt>“That’s your opinion,” Schmidt said, then joked: “If you could arrange a letter from Facebook and Twitter to us, that would be helpful.” sullivan> I pushed back that both have effectively given those le…

That doesn't make sense. Twitter's robots.txt allows profile pages and tweets to be indexed, and Google does index them. Google is simply choosing to exclude them from their people & places feature so that they can favor their own Google+ profiles. It's only the outbound links within tweets that Google is forbidden from indexing. That has nothing to do with why Google excludes content on twitter.com itself.

> That has nothing to do with why Google excludes content on twitter.com itself.

Um, they don't. That's not what this is about. You need to research the actual feature that's causing controversy, because it's not "Twitter doesn't show up in search", not even close.

Re: How rel=nofollow Works

#43

Post by mapgrep »

jerednel wrote: post #34

There seems to be some confusion about the different kinds of nofollow. Means not to follow the link to which the nofollow belongs. It may still be indexed, however, if the page is found by a either an internal or external followed link. The does not follow/crawl to any of the links on the page. These pages may be indexed, however if they are reached by either an internal or external followed link To keep the page ou…

"if Google wanted to, they could parse the external links from a Twitter stream without actually following them"

The problem is that there are no external links in the tweets. All tweets point to t.co, which is run by Twitter. You can't find out the ultimate destination of the link without following it through Twitter's redirector.

Re: How rel=nofollow Works

#44

Post by abalone »

nknight wrote: post #42
abalone wrote: post #39

Earlier quoted context omitted.

That doesn't make sense. Twitter's robots.txt allows profile pages and tweets to be indexed, and Google does index them. Google is simply choosing to exclude them from their people & places feature so that they can favor their own Google+ profiles. It's only the outbound links within tweets that Google is forbidden from indexing. That has nothing to do with why Google excludes content on twitter.com itself.

> That has nothing to do with why Google excludes content on twitter.com itself. Um, they don't. That's not what this is about. You need to research the actual feature that's causing controversy, because it's not "Twitter doesn't show up in search", not even close.

The People & Pages feature. On Google search. Is excluding twitter.com content.

Example: http://parislemon.com/post/15682237911/twitter-keeps-right-o...

Re: How rel=nofollow Works

#45

Post by lazerwalker »

jimmy_cheese wrote: post #6

But, MG Siegler is correct, is he not, that google have chosen not to link to artists twitter pages alongside their G+ pages, and that this could easily been seen as anti-competitive. e.g. https://www.google.com/search?&q=music

Yep, you're right.

There are two aspects to the new G+ search features: indicating that a link was shared by one of your friends, and the sidebar that links relevant G+ profiles. The former is presumably limited by rel=nofollow, but the latter has no relation to rel=nofollow.

If you search for "WWE", Google doesn't show you the @wwe Twitter account in the sidebar. The fact that Google is told not to follow any outgoing links posted by the @wwe account isn't a valid explanation for that, especially when you consider that (a) Google definitely has twitter.com/wwe indexed, and (b) twitter.com/wwe almost certainly has a high PageRank due to incoming links.

Re: How rel=nofollow Works

#46

Post by nknight »

abalone wrote: post #44
nknight wrote: post #42

Earlier quoted context omitted.

> That has nothing to do with why Google excludes content on twitter.com itself. Um, they don't. That's not what this is about. You need to research the actual feature that's causing controversy, because it's not "Twitter doesn't show up in search", not even close.

The People & Pages feature. On Google search. Is excluding twitter.com content. Example: http://parislemon.com/post/15682237911/twitter-keeps-right-o...

So what? That's not part of the standard search. Since when is Google not allowed to advertise its own services alongside standard search results? What's next, do you whine that Bing has links to MSN and Hotmail?

Twitter actually contaminates ordinary search results on its site with paid tweets, and you're mad Google puts some Google+ content off to one side? WTF?

Re: How rel=nofollow Works

#47

Post by Raphael »

mapgrep wrote: post #41
dolbz wrote: post #10

Earlier quoted context omitted.

The post you're linking is discussing the META tag version of nofollow, not the hyperlink rel attribute which is detailed here: http://googleblog.blogspot.com/2005/01/preventing-comment-sp... This article doesn't say explicitly that it won't follow those links and I would suggest it often will as comments are a rich source of pages to index. The nofollow on a link like this just indicates that the page owner can't vo…

"who are you to tell Google they can't index a third party domain?" No tweets link to third-party domains, since Twitter wraps all links in their own URL shortener (t.co). Even if you use an independent URL shortener, it will be wrapped by t.co. (I suppose you could argue that there is no way for Google to determine this algorithmically, since "twitter.com" != "t.co", so it should go ahead with the crawl, but there's…

They could create a robots.txt for t.co and set it to noindex.

Re: How rel=nofollow Works

#48

Post by jasonlotito »

masklinn wrote: post #23
jasonlotito wrote: post #18

Earlier quoted context omitted.

> the only promise Google does is that it will not transfer pagerank credit to the link's destination Google is fairly clear about this: http://support.google.com/webmasters/bin/answer.py?hl=en&#38... "How does Google handle nofollowed links? In general, we don't follow them. This means that Google does not transfer PageRank or anchor text across these links. Essentially, using nofollow causes us to drop the target l…

Interesting, so their implementation has broadly expanded upon the original meaning of the attribute (and the one standardized as a microformat), and in recommending this be broadly used to fight against spam they've paved the way for an interesting time for everybody else?

Broadly expanded? Standardized? Please.

I think the nofollow attributes intent was fairly straight forward from the beginning: to designate links that should not be accorded attention in search results. In 2005, this was merely via PageRank. Today, it's recommendations in search results.

Why would I want to see search results that shouldn't be given weight in a search result in a search result? Explain that to me.

Re: How rel=nofollow Works

#49

Post by abalone »

nknight wrote: post #46
abalone wrote: post #44

Earlier quoted context omitted.

The People & Pages feature. On Google search. Is excluding twitter.com content. Example: http://parislemon.com/post/15682237911/twitter-keeps-right-o...

So what? That's not part of the standard search. Since when is Google not allowed to advertise its own services alongside standard search results? What's next, do you whine that Bing has links to MSN and Hotmail? Twitter actually contaminates ordinary search results on its site with paid tweets, and you're mad Google puts some Google+ content off to one side? WTF?

Oh, it's definitely part of standard search now. Schmidt said Twitter content wasn't included because they didn't have "permission" to index it. That's false. They are purposefully excluding it -- a point of fact which I'm glad to see you acknowledge now.

Re: How rel=nofollow Works

#50

Post by abalone »

lazerwalker wrote: post #45
jimmy_cheese wrote: post #6

But, MG Siegler is correct, is he not, that google have chosen not to link to artists twitter pages alongside their G+ pages, and that this could easily been seen as anti-competitive. e.g. https://www.google.com/search?&q=music

Yep, you're right. There are two aspects to the new G+ search features: indicating that a link was shared by one of your friends, and the sidebar that links relevant G+ profiles. The former is presumably limited by rel=nofollow, but the latter has no relation to rel=nofollow. If you search for "WWE", Google doesn't show you the @wwe Twitter account in the sidebar. The fact that Google is told not to follow any outgoi…

Thank you. A lot of people are confusing these two aspects.

I'll add one more thing though: Schmidt actually claimed they didn't have "permission" to include Twitter and Facebook pages in the suggested profiles sidebar. That's just plain false.

Watch at about 1:10 in the video. http://marketingland.com/schmidt-google-not-favored-happy-to...

Post reply on HN