Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rubygems.org AWS bill for Feb 2014 [pdf]

dropbox.com

Post reply on HN
Print view View original

131–140 of 153 posts

Re: Rubygems.org AWS bill for Feb 2014 [pdf]

#131

Post by michaelmior »

jcampbell1 wrote: post #38
patio11 wrote: post #6

While one could probably knock a couple thousand bucks off that if one cared to (which is probably penny wise and pound foolish but invariably comes up in HN discussions of hosting costs), the amazing thing is that hundreds of thousands of people worldwide are able to use core infrastructure which costs less than the fully-loaded cost of a single billing clerk in your local municipal water department.

What is funny is that Github is footing the bill for most package systems, which were likely inspired by ruby gems, yet Github itself was built with Ruby gems. I am pretty sure the hosting costs for homebrew/npm round to nil (I could be wrong).

Why do you say GitHub is footing the bill? I understand for Homebrew since it uses GitHub repositories, but npm is served via a CouchDB instance which I believe is sponsored by Joyent.

Re: Rubygems.org AWS bill for Feb 2014 [pdf]

#132

Post by ceejayoz »

hackerboos wrote: post #125
Zarel wrote: post #99

Earlier quoted context omitted.

My site serves 25 TB/mo, and it costs me $80/mo... $60/mo for a dedicated server, $20/mo for CloudFlare. The dedicated server only serves 1 TB of it, the other 24 TB is static assets cached and served directly by CloudFlare. Here's a screenshot of CloudFlare Analytics for the last 30 days: http://d.pr/i/6Z8S/5GU2Ni8t

CloudFlare cannot be making money on you as a customer. $20 for 24TB is too cheap.

It's offset by the many customers paying $20 and using 1 GB, I'd imagine.

Re: Rubygems.org AWS bill for Feb 2014 [pdf]

#133

Post by ceejayoz »

ne0lithic wrote: post #40

With most of this being bandwidth costs, it seems like switching to a host like Digital Ocean would make more sense here. The bandwidth costs are a fraction of Amazon's in comparison. As for the CDN, switching to something like Cloudflare might make more sense rather than relying on Cloudfront. At the least, there's a "US and EU only" option for edge locations to use which si considerably cheaper than the default opt…

> With most of this being bandwidth costs, it seems like switching to a host like Digital Ocean would make more sense here. The bandwidth costs are a fraction of Amazon's in comparison.

That's just replacing bandwidth costs with build-and-run-your-own-CDN costs.

Re: Rubygems.org AWS bill for Feb 2014 [pdf]

#134

Post by ceejayoz »

fivethree wrote: post #109
duaneb wrote: post #8

Earlier quoted context omitted.

> which costs less than the fully-loaded cost of a single billing clerk in your local municipal water department. To be fair, a lot of maintenance value goes into the software that is never quantified. Broken software breaks hard, not partially, so maintenance is even more crucial.

When a levy breaks people die. Software maintenance and damage is nothing compared to real engineering.

> Software maintenance and damage is nothing compared to real engineering.

http://en.wikipedia.org/wiki/Cluster_(spacecraft) cost $370 million when an overflow caused a rocket to explode.

I'd imagine there's some mission critical software running nuclear plants, aircraft, cars, etc.

Re: Rubygems.org AWS bill for Feb 2014 [pdf]

#135

Post by jonaldomo »

_petronius wrote: post #116
fivethree wrote: post #109

Earlier quoted context omitted.

When a levy breaks people die. Software maintenance and damage is nothing compared to real engineering.

That may be true of web development, but certainly isn't of software as a whole: http://en.m.wikipedia.org/wiki/Therac-25

I disagree with the web development comment. What if there was a web interface on top of Therac-25 that had an error in it?

Re: Rubygems.org AWS bill for Feb 2014 [pdf]

#136

Post by JohnBooty »

Mister_Snuggles wrote: post #44
adventured wrote: post #42

Earlier quoted context omitted.

Serious question: muni water dept billing clerks make over $84,000 per year all in?

Once you factor in employer-paid benefits, pension, etc, the employer's cost is something like 140% of the employee's gross pay. This turns the $84,000 into a $60,000 gross salary. If you include other costs, like the office space and equipment used by the employee, it starts to sound pretty reasonable.

So true! Also, in most organizational structures, every X employees will need one manager, which is also a cost that needs to be factored into the cost of hiring an employee.

Re: Rubygems.org AWS bill for Feb 2014 [pdf]

#137

Post by joshcrowder »

jasonkester wrote: post #108
grey-area wrote: post #98

Earlier quoted context omitted.

It would be really interesting to see the bandwidth broken down by gem - I suspect rails would be at the top, but it'd be interesting to see. If most of the installs are on servers, have you considered talking to server providers about setting up internal mirrors on their networks? That might save everyone a lot of bandwidth. Of course, people shouldn't really be installing their gems from ruby gems on servers anyway…

Hey, a chance to plug my thing! I built S3stat ( https://www.s3stat.com/ ) to fix this opaqueness that comes with using Cloudfront as a CDN and get you at least back to the level of analytics you'd get if you were hosting files from one of your own servers. RubyGems guys, if you have logging set up already, I'd be happy to run reports for all your old logs (gratis, naturally) so you can get a better idea of which fil…

Off topic: S3stat is our go to service we've been using it for years and really couldn't survive without it as its how we charge our clients!

Re: Rubygems.org AWS bill for Feb 2014 [pdf]

#138

Post by iloveponies »

evanphx wrote: post #58
knyt wrote: post #55

Earlier quoted context omitted.

Did you consider using a mirror network, with servers run by external organizations, instead of going with AWS bandwidth for rubygems? Seems like that would be a good approach for the static/bulk part of your dataset, and there are lots of companies and universities who are set up to serve software. (The mirror I manage serves about 50 TB/month for several Linux distros, and many sites are larger.) Do the work and in…

It's been discussed many times before, yes. Rubygems usage pattern by our users make any kind of mirror delay unacceptable. We currently run a number of mirrors, configured as caching proxies. I want to get us going on a CDN like Fastly soon because they provide effectively the same functionality but distributed to many, many more POPs that I will ever setup.

I suspect mirror delay is less of an issue than you might perceive it to be. Many CPAN mirrors manage to stay within tens of seconds/no more than a minute from the main CPAN mirror that PAUSE publishes to.

Re: Rubygems.org AWS bill for Feb 2014 [pdf]

#140

Post by fivethree »

ceejayoz wrote: post #134
fivethree wrote: post #109

Earlier quoted context omitted.

When a levy breaks people die. Software maintenance and damage is nothing compared to real engineering.

> Software maintenance and damage is nothing compared to real engineering. http://en.wikipedia.org/wiki/Cluster_(spacecraft) cost $370 million when an overflow caused a rocket to explode. I'd imagine there's some mission critical software running nuclear plants, aircraft, cars, etc.

I'll agree mission critical software exists. I however imagine there are far more engineering projects across the planets whose failure result in mass casualties than software. There is a reason actual engineers are legally liable for their work.
Post reply on HN