Live data from Hacker News

Bit.ly is Harmful to Your Reputation

cranialsoup.blogspot.com

41–50 of 118 posts

Re: Bit.ly is Harmful to Your Reputation

#41
post #26
post #10

Earlier quoted context omitted.

Extra cloaking for malicious links? Bit.ly can't exactly check the original site to see if it's OK if it's hidden behind a funky redirection layer. I think that bit.ly was reasonable to have an interstitial, although maybe they should be more open about why and how to remove it.

It's a standard http 301 redirect, not a "funky redirection layer". /Users/sjs % curl -i http://xrl.in/33qj HTTP/1.1 301 Permanent Redirect Date: Sun, 09 May 2010 00:38:19 GMT Server: Apache Location: http://www.donationcoder.com/CodingSnacks/index.php Content-Length: 0 Content-Type: text/html

Right, but the problem is this scenario:

Bit.ly see's your xrl.in and does a request. They find 301 and the location at donationcoder.com. They conclude "this site is ok". Later, the xrl.in url is changed to .

They aren't going to do a request to every url they're linking to on every click, obviously. So they'd only get the one chance.

Now, I'm not actually sure that xrl.in lets you change links after shortening. The point is that bit.ly doesn't know either.

Re: Bit.ly is Harmful to Your Reputation

#42
post #39

Does SMS have to display the href? Could Twitter just replace all URL's with the word 'link' and show the real URL on hover, thereby doing away with URL shorteners altogether?

Hover doesn't work well with touch based interaction, which is all the rave these days, I hear. I like the idea of showing them separately somehow, though.

Re: Bit.ly is Harmful to Your Reputation

#44
Todd from bit.ly here.

From day one, we've prized security, transparency, reliability, and openness at bit.ly. Along the way, we've made a number of product decisions based on those tenets. Among those are link permanence (link destinations don't change once created), the avoidance of anything that interferes with user experience (we've never framed, nor will we), and a dedicated focus on spam and malware detection, so that our users can click on bit.ly links with a high degree of confidence.

We take our responsibility as internet citizens seriously, and you'll see this exhibited even in the small details of the ways in which we manage flagged links (you'll notice we never actually disable a redirect, and at most simply insert an interstitial which retains the end destination link).

In the course of analyzing content for spam, malware, and phishing attacks, we rely on a number of systems, both internal and external. Over the course of the past year, a number of spammers have attempted to use various levels of indirection through redirectors (some of which are reconfigurable), in order to obfuscate and cloak their efforts. In fact, the bulk of shortens to bit.ly coming through other URL shorteners have tended to be attempts to spam the system. While our crawlers do of course follow links through redirections, the inclusion of modifiable redirects in the stream, and our analysis of the preponderance of spam attempts via these vectors have made it necessary and appropriate in some cases to block the URL shorteners.

Just to reiterate, the only goal is and always has been to protect the end user clicking on bit.ly links, regardless of the link source. Given that multiple layer wrapping of URL redirectors tends to be an edge case based on inappropriate API usage, confused users, or in the preponderance of cases, attempts to spam, we think this has been a fair approach. As such, you'll note that we did in fact update our interstitial warning pages with language better reflecting the reasoning behind the status. We're happy to see a healthy, vibrant, shortening ecosystem, and have no intention whatsoever to put a damper on other sites in the space.

Some have suggested we simply not shorten URLs already pointing to 3rd party short URLs. While this is a potential possibility, our API responses and the innumerable clients and scripts that use these methods aren't currently designed with this state in mind. Consequently, any changes would have to be carefully considered.

As with any product, bit.ly is a work in progress, and we're always interested in finding ways to best serve our users, while maintaining the integrity and openness of the product.

Re: Bit.ly is Harmful to Your Reputation

#45
post #32

Earlier quoted context omitted.

They could apply URL shortening to a tweet when delivering over SMS, if necessary.

Or they could apply URL canonicalization to tweets delivered any other way: follow all the 302s until they get the actual page, then replace with those URLs instead. (Actually, the best approach would be to canonicalize the URL on its way in, and then shorten with a single, known URL shortener on the way out, if sending via SMS. This would stop the "shortener wars" in its tracks, which is probably a good thing, becau…

and then shorten with a single, known URL shortener on the way out

Which should be Twitter's own built-in shortener.

Re: Bit.ly is Harmful to Your Reputation

#46
post #44

Todd from bit.ly here. From day one, we've prized security, transparency, reliability, and openness at bit.ly. Along the way, we've made a number of product decisions based on those tenets. Among those are link permanence (link destinations don't change once created), the avoidance of anything that interferes with user experience (we've never framed, nor will we), and a dedicated focus on spam and malware detection,…

You say "any changes would have to be carefully considered". So, are said changes being considered or are they not?

Re: Bit.ly is Harmful to Your Reputation

#47
post #44

Todd from bit.ly here. From day one, we've prized security, transparency, reliability, and openness at bit.ly. Along the way, we've made a number of product decisions based on those tenets. Among those are link permanence (link destinations don't change once created), the avoidance of anything that interferes with user experience (we've never framed, nor will we), and a dedicated focus on spam and malware detection,…

> While our crawlers do of course follow links through redirections, the inclusion of modifiable redirects in the stream, and our analysis of the preponderance of spam attempts via these vectors have made it necessary and appropriate in some cases to block the URL shorteners.

Why not do as tkaemming suggests and follow the redirections to link to the final endpoint URL?

Re: Bit.ly is Harmful to Your Reputation

#48
post #46
post #44

Todd from bit.ly here. From day one, we've prized security, transparency, reliability, and openness at bit.ly. Along the way, we've made a number of product decisions based on those tenets. Among those are link permanence (link destinations don't change once created), the avoidance of anything that interferes with user experience (we've never framed, nor will we), and a dedicated focus on spam and malware detection,…

You say "any changes would have to be carefully considered". So, are said changes being considered or are they not?

We always consider ways in which to provide the best level of service to our end users and API users, while preventing unintended side effects. There are pros and cons to every approach, and they have to be evaluated with care.

Re: Bit.ly is Harmful to Your Reputation

#49
post #44

Todd from bit.ly here. From day one, we've prized security, transparency, reliability, and openness at bit.ly. Along the way, we've made a number of product decisions based on those tenets. Among those are link permanence (link destinations don't change once created), the avoidance of anything that interferes with user experience (we've never framed, nor will we), and a dedicated focus on spam and malware detection,…

> While our crawlers do of course follow links through redirections, the inclusion of modifiable redirects in the stream, and our analysis of the preponderance of spam attempts via these vectors have made it necessary and appropriate in some cases to block the URL shorteners. Why not do as tkaemming suggests and follow the redirections to link to the final endpoint URL?

The intent behind the very statement you quoted was to convey that we do precisely that. However, also mentioned was the fact that in a number of cases, modifiable destination redirects are embedded within the chain. In those cases, unless the redirect is crawled on every clickthrough, the integrity of the chain is difficult to assert.

Re: Bit.ly is Harmful to Your Reputation

#50
post #26

Earlier quoted context omitted.

It's a standard http 301 redirect, not a "funky redirection layer". /Users/sjs % curl -i http://xrl.in/33qj HTTP/1.1 301 Permanent Redirect Date: Sun, 09 May 2010 00:38:19 GMT Server: Apache Location: http://www.donationcoder.com/CodingSnacks/index.php Content-Length: 0 Content-Type: text/html

Right, but the problem is this scenario: Bit.ly see's your xrl.in and does a request. They find 301 and the location at donationcoder.com. They conclude "this site is ok". Later, the xrl.in url is changed to . They aren't going to do a request to every url they're linking to on every click, obviously. So they'd only get the one chance. Now, I'm not actually sure that xrl.in lets you change links after shortening. The…

Bit.ly could check a link to no_malware_here.com, which thereafter adds a 301 that redirects to your_computer_now_has_aids.com.

I think it is misleading to display that message based on the possibility of a redirection. Any page can do that, not just xrl.in.

Post reply on HN