221–230 of 311 posts
Convenient that we're too backward to use AWS. That means everyone can at least talk about it here when AWS is down.
Funny thing is, the last couple interviews I've had in Chicago and Silicon Valley, I actually get points when explaining caution is necessary when using AWS for production. A magic bullet it isn't.
Earlier quoted context omitted.
This kind of thinking is poisonous. I know it's in good fun and it's fun to look for connections in things, but it is actually preposterous to think that Amazon would purposely disrupt wide swaths of highly paying customers for much of a day to bury one story about bad customer relations. My guess is there are a lot of people working very hard to try and solve this problem right now, let's not belittle their efforts…
It was a joke. I made the same joke earlier today. No-one is seriously going to believe this.
Earlier quoted context omitted.
Until it can't. Then five years later you get Spanner.
You can buy some seriously big boxes, and easily split off a lot of services onto multiple boxes. The big problem with the "single big box" strategy is being able to do upgrades -- I see hn go down frequently for 5-10 min at a time in the middle of the night, which I assume is upgrades/reboots. The happy medium is probably splitting database (master/slave at least) and cdn (if needed) and some other services (AAA? lo…
For most applications I think architecting EBS out should be straightforward - instance storage doesn't put a huge single-point-of-failure in the middle of your app if you're doing replication, failover, and backups properly.
And EBS seems to be the biggest component of the recent AWS failures upon which they've built a lot of their other systems.
Convenient that we're too backward to use AWS. That means everyone can at least talk about it here when AWS is down.
I wish that DNS could just switch over to another availability zone when this happens. A datacenter with all the sql replicated. Sure you'd pay twice as much for EBS, but it could also double as a backup. As for the other resources at the availability zone, they are hardly utilized until they spring into action (EC2, etc.)
Earlier quoted context omitted.
To be fair, AWS downtime always make the news because they affect a lot of majors websites, but that doesn't mean an average sysadmin (or devops, whatever) would do better in term of uptime with his own bay and his toys.
When our gear is down, we can actually get into the datacenter to fix it. What do you do when Amazon is down other than sweat?
2. Calculate the cost of moving to the Oregon AWS datacenter.
3. Reassure your investors that outsourcing non-core competencies is still the way to go.
4. Try to, er, control, your inner control-freak.
;)
Earlier quoted context omitted.
I'm surprised amazon hasn't built another region in the east. If you're in the west you get US-West-1 and US-West-2 and can failover and distribute between the two, why don't they have that kind of duplication in the east?
Stop thinking about regions as datacenters. us-east-1 was 11 different datacenters last time I bothered to check. us-west-2 by comparison is two datacenters. The reason west-1 and west-2 exist is because they are geographically diverse enough to prevent low latency inner-connections (and also have dramatically different power costs so they bill differently).
Earlier quoted context omitted.
The last Netflix post mortem mentioned they had a bug in their configuration where they kept sending traffic to already down ELB instances, which was the cause of the last outage for them if I remember correctly.
AirBnB is also down.
Earlier quoted context omitted.
Funny thing is, the last couple interviews I've had in Chicago and Silicon Valley, I actually get points when explaining caution is necessary when using AWS for production. A magic bullet it isn't.
Caution and understanding is necessary when choosing any infrastructure provider; AWS is not a special case.