Chris Pirillo’s CDN in a Box looks to make Web sites hum at ‘Google-like speeds’ • Hacker News Board

Re: Chris Pirillo’s CDN in a Box looks to make Web sites hum at ‘Google-like speeds’

#11

Post by drakaal » Fri, May 27, 2011, 7:27 PM UTC

samuel1604 wrote: post #4

and how different from akamai dynamic site accelerator : http://www.akamai.com/html/technology/products/dsa.html

DSA is great. Better in some instances than CDNinaBox.com if you have the right traffic patterns. DSA leverages 50+ Points of presence however. And until your content is requested at each PoP it doesn't exist there. So if a visitor hits yourdomain.com/awesomeness and then the Google Ad bot hits the same page with DSA unless the user and the adbot came from the same-ish place you still get 2 hits on your server. With CDN In a Box there are multiple cache tiers so if you get 2 requests from the same place you get Super Mega Turbo (that's the technical name) response, if they are farther apart you only get Mega Turbo, and if they are really far apart you only get turbo. But you still only put the load of 1 request on your server, per cache interval. Akamai is better if you have an event you KNOW is going to be big, can plan for the capacity, and has a finite number of pages you need cached. CDNInABox.com could handle the traffic for the Indy500 or Kentucky derby, but Akamai would be a better choice. We aren't attempting to be all things to all people. But our lower barrier to entry, and our greater ease of installation makes us a great choice for a large segment of the internet that aren't in the right class for DSA.

Re: Chris Pirillo’s CDN in a Box looks to make Web sites hum at ‘Google-like speeds’

#12

Post by drakaal » Fri, May 27, 2011, 7:36 PM UTC

eastdakota wrote: post #9

drakaal wrote: post #8

Earlier quoted context omitted.

We don't block the Robots, so you don't get delisted by Google for using the service. That'd be a big difference.

CloudFlare doesn't block legit crawlers either. It does cache responses to crawlers so if a page hasn't changed and Google crawls it again the request doesn't burden the origin. What's interesting about CDN in a Box is they're serving off a single IP. The problem with this strategy is Google classifies sites for crawl purposes by IP. That means if one site on CDN in a Box falters, all the other sites on CDN in a Box…

I have personally had to get sites that use CloudFlare re-listed after being booted from Adsense, or Google because Cloud Flare served a different page to bots than to users and kicked off the Access Restricted Page.

You keep saying we serve off of one IP that is blatantly false.

I'll put my Crawl Rate Up Against anyone's because we had to have a conversation with Google's team because their Bot hit one of our sites for 1.2M crawled pages in 3 hours. Which is nice, but then they did it again the next day. and the next. So we are negotiating to not have to pay for Google bot traffic.

In webmaster's tools you can't even change the setting of a CDN In A Box Site, because Google Assigns you a Special Crawl rate.

Bing's Bot loves us, because they will often crawl "all pages at once" so they will crawl 10k pages in 30 seconds. and go on to the next site.

Re: Chris Pirillo’s CDN in a Box looks to make Web sites hum at ‘Google-like speeds’

#13

Post by drakaal » Fri, May 27, 2011, 7:49 PM UTC

eastdakota wrote: post #9

drakaal wrote: post #8

Earlier quoted context omitted.

We don't block the Robots, so you don't get delisted by Google for using the service. That'd be a big difference.

CloudFlare doesn't block legit crawlers either. It does cache responses to crawlers so if a page hasn't changed and Google crawls it again the request doesn't burden the origin. What's interesting about CDN in a Box is they're serving off a single IP. The problem with this strategy is Google classifies sites for crawl purposes by IP. That means if one site on CDN in a Box falters, all the other sites on CDN in a Box…

Also I confirmed with Google:

"Because of the propensity of shared hosting providers, the recycling of IP addresses by cloud services, and the ease at which IP addresses can be changed, we never use IP address as a factor for assigning a penalty to a website. In certain instances where malware is hosted on a site, we may display the "this site may be harmful" warning against all results on an IP address. The rules for when this happens are not my expertise, but from the prospective of search, your site will never receive a penalty for sharing an IP with a blocked, banned, or delisted site."