91–100 of 153 posts
Earlier quoted context omitted.
Could probably fix some of this just by talking to Amazon about it. It's not like this is a 'for profit' setup.
Exactly, and if you include a little Powered by AWS CloudFront i am pretty sure they could drive down the price a lot. Or, they could start talking to Fastly, I am pretty sure they can work out a much better deal while being faster.
Earlier quoted context omitted.
Really, why would that be the case? I mean the difference in Latency. A self set up Linode CDN with all six location Tokyo, JP London, UK Newark, NJ Atlanta, GA Dallas, TX Fremont, CA Would have provided 48TB of Pooled bandwidth at a very decent speed and cost around $480. Linode's Network are great, much better then DO. I am not sure if it match CloudFront, which isn't exactly the fastest CDN anyway.
Having volunteers spend countless hours to setup and maintain their own CDN is a preposterous idea.
http://psyphi.net/blog/2013/12/content-delivery-network-cdn-...
I have 10 fingers, so that is definitely not "countless" hours of work. And No, Maintenance are minimal or non existent. You could even get smaller VPS behind each node balancer as HA. Since Linode VPS ( Unlike DO ) are deployed on physically different hardware.
While i say it is fair enough to use AWS because money doesn't matter, i thought there are definitely some better alternative for the same price( if you really cared about latency ) or cheaper options.
Earlier quoted context omitted.
But they are just using Amazon's Cloudfront. They aren't using SQS or anything. What technology risk is there in setting up Varnish and nginx on Digital Ocean? Or better yet some kind of out-of-the-box open source CDN. You would save a lot more than $2-3k.
I'm referring to Digital Ocean's technology. AWS (and Media Temple) had it's teething pains as well in it's first 4-5 years. I remember some fairly broad outages with their back end storage - but they've mostly dealt with those, and, the risk has gone down. Note - there is another option - Deploy on multiple Platforms and be smart with your DNS balancing ( http://www.dnsmadeeasy.com/services/global-traffic-director/…
Would love to see any recent input.
Earlier quoted context omitted.
I think that bandwidth consumed by Circle should be free, since we're also hosted in AWS. Maybe somebody who knows more about the details of Amazon's billing can confirm/deny.
bandwidth is free in the same region, but not across regions. *edit and I believe not if you end up using the public IP address instead of the internal ip address.
Earlier quoted context omitted.
Thanks! I'll bring it up with the team.
Note that if you can get Rackspace or whomever to donate the hardware/bandwidth, you can use less than 7k/month to hire a very competent admin to solve the administrative issues, which would probably lead to better service for everybody.
With most of this being bandwidth costs, it seems like switching to a host like Digital Ocean would make more sense here. The bandwidth costs are a fraction of Amazon's in comparison. As for the CDN, switching to something like Cloudflare might make more sense rather than relying on Cloudfront. At the least, there's a "US and EU only" option for edge locations to use which si considerably cheaper than the default opt…
It's possible RubyGems.org would be classified under one of the "not really allowed here" terms.
With most of this being bandwidth costs, it seems like switching to a host like Digital Ocean would make more sense here. The bandwidth costs are a fraction of Amazon's in comparison. As for the CDN, switching to something like Cloudflare might make more sense rather than relying on Cloudfront. At the least, there's a "US and EU only" option for edge locations to use which si considerably cheaper than the default opt…
I thought I'd answer some of your questions, as the person that pays the bill. 1. This can be cheaper on AWS. We've been meaning to move to reserve instances, paying a year at a time, for a while and simply haven't done it yet. 2. Fastly has already donate CDN usage to us, but we haven't fully utilized it yet as we're (slowly) sort out some issues between primary gem serving and the bundler APIs. 3. RubyCentral pays…
If most of the installs are on servers, have you considered talking to server providers about setting up internal mirrors on their networks? That might save everyone a lot of bandwidth.
Of course, people shouldn't really be installing their gems from ruby gems on servers anyway, is there any way to prod bundler to make it default to package gems and do a local install where possible, rather than downloading them every time there is a deploy (the current default)? At present you use double bandwidth from people downloading once on their local machine, and once on their server to update.
Fetching the ruby gems index with bundler/rubygems still takes a while every time I bundle update, have you looked at optimising that part of the process further (at least it doesn't fetch a list of all gems now, but it still fetches a list of all versions of each gem doesn't it?), say caching older gem results? The list of gem versions available should not change for old ones, so you should really only need to fetch a very small list of latest versions. The memory usage and bandwidth usage is still quite high there.
The thing is there are many providers who can do the same and most of them will do it for less than half of this. Some less than 1/5th. I think they should move this to Digital Ocean and save $5000. The bias towards AWS for this type of application is ridiculous and a big waste of money.
Whenever anybody makes this type of statement, I'm alway interested in knowing if they've ever run a site with this type of traffic, and this many customers. In particular, have you ever run a site that consistently serves over 25 Terabytes of traffic/month, or have you worked with someone who has? I guarantee you that no company I have worked for in the last 15 years, could have ever run this type of infrastructure…
$60/mo for a dedicated server, $20/mo for CloudFlare. The dedicated server only serves 1 TB of it, the other 24 TB is static assets cached and served directly by CloudFlare.
Here's a screenshot of CloudFlare Analytics for the last 30 days: http://d.pr/i/6Z8S/5GU2Ni8t
CloudFront $1,071 Data Transfer $3,597 EC2 $2,184 S3 $ 228
While "bandwidth" costs equate to ~$4,668/month, only $1,071 is CDN (CloudFront), with the balance just raw Data Transfer.
Since lots of folks are commenting, and not everyone realizes the difference it's also a good time to point out the CloudFront vs. Data Transfer distinction.
Using Amazon's terms... Data Transfer means anything directly served/coming from EC2 or S3 (or a few other services which aren't relevant here), but NOT anything for CloudFront (which is, obviously, a separate line item, as shown above).
The bulk of CDN (CloudFront) usage ($735 worth or 69%) is US.
The bulk of Data raw bandwidth (Data Transfer) usage ($2,931 ~80%) is US East.