Earlier quoted context omitted.
Do you know who are the biggest consumers of bandwidth? I would guess the CI servers (Travis, Circle)
A very good question. I'll see about crunching some of the logs to break it down by subnet.
Rubygems.org AWS bill for Feb 2014 [pdf]
111–120 of 153 posts
Re: Rubygems.org AWS bill for Feb 2014 [pdf]
#112The thing is there are many providers who can do the same and most of them will do it for less than half of this. Some less than 1/5th. I think they should move this to Digital Ocean and save $5000. The bias towards AWS for this type of application is ridiculous and a big waste of money.
> The bias towards AWS for this type of application is ridiculous and a big waste of money. They could get an even better deal by just going through a dedicated server provider (or even better, colocating). There's little advantage with choosing DO versus going with a dedicated server provider (and again, colocating). I guess the advantage would be the control panel that they wouldn't use, having a few one-click stac…
I don't know what datacentres tend to charge for data transfer, but as that's the largest item on the bill, it's the more salient point.
Also, just because it's not on the bill doesn't mean they're not using other AWS services; there are several free ones.
Re: Rubygems.org AWS bill for Feb 2014 [pdf]
#113Earlier quoted context omitted.
You miss the point; bandwidth from a single source != CDN. Hosting without a CDN would be extremely slow for people that were far (hops) away from DO. CDN's solve this, and other issues.
How much does that really matter? Even going to the otherside of the world is only 200ms or so, and the time taken to run rubygems is hardly a factor in just about any workflow I can imagine. Think of it another way - what would be more valuble - RubyGems hosted on CDN, or RubyGems on DO and give a couple of grants for talent hackers to work on their gems fulltime for a few months. (ala GSoC) Even if you ARE concerne…
Re: Rubygems.org AWS bill for Feb 2014 [pdf]
#114Earlier quoted context omitted.
I still don't understand how AWS was preferable to a dedicated server host. Could you elaborate on that?
Virtualization allows us to spin up new instances and migrate traffic to them. This means we can work entirely from chef and keep things clean. This is important for our volunteers to have a complete picture of an instance and to be able to make new ones.
And pretty much "all" dedicated server providers these days also have cloud offerings if you need to spin up some instances quickly to handle traffic spikes etc., or for dev/testing purposes.
Re: Rubygems.org AWS bill for Feb 2014 [pdf]
#115I thought I'd answer some of your questions, as the person that pays the bill. 1. This can be cheaper on AWS. We've been meaning to move to reserve instances, paying a year at a time, for a while and simply haven't done it yet. 2. Fastly has already donate CDN usage to us, but we haven't fully utilized it yet as we're (slowly) sort out some issues between primary gem serving and the bundler APIs. 3. RubyCentral pays…
Re: Rubygems.org AWS bill for Feb 2014 [pdf]
#116Earlier quoted context omitted.
> which costs less than the fully-loaded cost of a single billing clerk in your local municipal water department. To be fair, a lot of maintenance value goes into the software that is never quantified. Broken software breaks hard, not partially, so maintenance is even more crucial.
When a levy breaks people die. Software maintenance and damage is nothing compared to real engineering.
Re: Rubygems.org AWS bill for Feb 2014 [pdf]
#117Earlier quoted context omitted.
> which costs less than the fully-loaded cost of a single billing clerk in your local municipal water department. To be fair, a lot of maintenance value goes into the software that is never quantified. Broken software breaks hard, not partially, so maintenance is even more crucial.
When a levy breaks people die. Software maintenance and damage is nothing compared to real engineering.
Re: Rubygems.org AWS bill for Feb 2014 [pdf]
#118Earlier quoted context omitted.
> which costs less than the fully-loaded cost of a single billing clerk in your local municipal water department. To be fair, a lot of maintenance value goes into the software that is never quantified. Broken software breaks hard, not partially, so maintenance is even more crucial.
When a levy breaks people die. Software maintenance and damage is nothing compared to real engineering.
Software controls everything from nuclear power stations to missles to dams to radiation therapy machines (where, again, software killed 3 people - http://en.wikipedia.org/wiki/Therac-25)
Proper software engineering is increasingly more important and, I'd posit, likely to become even more important than civil engineering for public safety as time goes on.
Re: Rubygems.org AWS bill for Feb 2014 [pdf]
#119Earlier quoted context omitted.
This bill is 2/3 bandwidth, and 1/3 compute. Some games require massive amounts of compute, but the bandwidth to deliver the assets is generally paid by Apple. I can guarantee you, your company is paying a metric fuck-ton more. It is called Apple's 30% cut. Your company is paying AWS $200k to pass json messages around for analytics and social aspects of the game. You are paying Apple something like $1 million per wee…
We aren't paying that much in cut just yet. We're a small team (6 engineers in total). You don't have to be pulling in millions per week to get high on the grossing charts. We're probably around 1/4 of what you estimated. Though you mention delivering the assets. Actually (like a lot of games) we make a big effort in getting under 50MB over-the-air limit on the App Store. The total content for retina iPhone is ~300MB…
We're managing to squeeze our apps into this at the moment, but will likely need a similar solution using S3/CloudFront in the near future.
[1] http://www.macrumors.com/2013/09/18/apple-increases-over-the...
Re: Rubygems.org AWS bill for Feb 2014 [pdf]
#120While by no means insignificant, this bill is no where near what I'd imagine would warrant a HN post. I wouldn't be surprised if most startups beat this regularly. The startup whose backend I co-created racks up an AWS bill that hovers around a half million dollars a month. We make use of all of the ways to save with Amazon: pre-paid reserved instances, negotiated deals, etc. And we're not even that big; imagine what…
> this bill is no where near what I'd imagine would warrant a HN post. Can only guess at why folks like any post, but it's not necessarily how large the bill is. Maybe it's how low it is for a service that's widely relied on, or maybe it's the level of transparency, which turned out to include evanphx above showing up to answer questions about the project.
Compared to npm asking for $300,000 in donations to keep the thing running. I'm glad RubyGems can run for relatively so little, and be transparent in doing so.