http://venuetastic.com/ - feel bad for these guys. They launched yesterday and down today because of AWS. Murphy's law in practice.
Amazon Web Services are down
281–290 of 346 posts
Re: Amazon Web Services are down
#282Earlier quoted context omitted.
That hurts. But you're right, we've had a lot of issues. I think the reason this is news is because it is a massive Amazon failure.
I can't imagine how frustrating the jobs of the Reddit admins must be.
Frankly, for the size of the site, they do really, really well for the limited resources they have.
Re: Amazon Web Services are down
#283We're experiencing problems with two of our ELBs, one indicating instance health as out of service, reporting "a transient error occurred". Another, new LB (what we hoped would replace the first problematic LB), reports: "instance registration is still in progress".
A support issue with Amazon indicated that it was related to the ongoing issues and to monitor the Service Health Dashboard. But, as I mentioned before, ELB isn't mentioned at all.
Re: Amazon Web Services are down
#284A couple of hours into the failure, and no sign of coverage on Techcrunch (they're posting "business" stories though). It shows how detached Techcrunch has become from the startup world. Edit: I tweeted their European editor about it and he's posted a story up now.
Perhaps this isn't really news. These days it's normal.
Re: Amazon Web Services are down
#285Some quotes regarding how Netflix handled this without interruptions: "Netflix showed some increased latency, internal alarms went off but hasn't had a service outage." [1] "Netflix is deployed in three zones, sized to lose one and keep going. Cheaper than cost of being down." [2] [1] https://twitter.com/adrianco/status/61075904847282177 [2] https://twitter.com/adrianco/status/61076362680745984
Last 60 minutes comparison data: http://dl.dropbox.com/u/1898990/EC2-multiple-zones-60m.png
time in GMT.
A study we (Cedexis) did in January comparing multiple ec2 zones and other cloud providers: (pdf) http://dl.dropbox.com/u/1898990/76-marty-kagan.pdf
Re: Amazon Web Services are down
#286Re: Amazon Web Services are down
#287Earlier quoted context omitted.
http://cache-scale.appspot.com/c/www.reddit.com/ That's a cache of it. I really wish that the admins at reddit would implement something like this themselves, then link to it when downtime like this happens.
do you have any stats of this app in terms of total data stored, bandwidth required per day, requests per second?
Re: Amazon Web Services are down
#288Just launched: http://techcrunch.com/2011/04/21/page-sharing-service-bo-lt-...
Re: Amazon Web Services are down
#289Earlier quoted context omitted.
Apparently most of their problems are caused by bad EBS writes/performance, or at least so they said a few weeks ago after some particularly bad downtime. It looks like EBS will randomly decide to switch to a few bps of performance from time to time. I would use Amazon for my startup, but these issues really make it hard to justify.
EC2 instances only have one network interface. The public IP address you have pointing to your instance is a DNAT done somewhere further up the chain. If you get a large network load to your instance - say, a DDoS attack - you can find you no longer have enough network capacity to talk to your EBS disks. This is what happened to Bitbucket in 2009: http://blog.bitbucket.org/2009/10/04/on-our-extended-downtim...
Re: Amazon Web Services are down
#290Earlier quoted context omitted.
http://cache-scale.appspot.com/c/www.reddit.com/ That's a cache of it. I really wish that the admins at reddit would implement something like this themselves, then link to it when downtime like this happens.
They do have a read-only mode, don't they? I'm not sure why they don't enable read-only mode when things like this happen. It may be that Amazon's service being down forbids this. I dunno.