261–270 of 646 posts
Coworkers: "You're an f'n idiot. Amazon and Facebook don't go down, you're holding us back!" Me: leaves cause that treatment was the final straw
Amazon and Facebook both go down within a month of each other, and supposedly they needed backups
Them: shocked pikachu face
Ok, enough AWS outages to say I'm tired of hearing about low end stuff being flaky.
"Don't use a self hosted monolithe, it's not reliable! You need a cloud FS with a load balancer under observability and your data in a db that scales horizontally, all orchestrated by kubs." Meanwhile, I currently have a gig to work on a video service which features a never updated centos 6, an unsupported python 2 blob website, and a push to prod deployment procedure, running a single postgres db serving streaming f…
I've built out many 42U racks in DC's in my time and there were a couple of rules that we never skipped: 1. Dual power in each server/device - One PSU was powered by one outlet, the other PSU by a different one with a different source meaning that we can lose a single power supply/circuit and nothing happens 2. Dual network (at minimum) - For the same reasons as above since the switches didn't always have dual power…
> I've only had a DC fail once when the engineer was performing work on the power circuitry for the DC and thought he was taking down one, but was in fact the wrong one and took both power circuits down at the same time. This is all local scale. Your setup would not survive a data center scale power outage. At scale power outages are datacenter scale. Data centers lose supply lines. They lose transformers. Sometimes…
Citation needed - the same issue with testing, data races and expensive bandwidth come up.
If you haven't seen yet, news is it was a power loss: > 5:01 AM PST We can confirm a loss of power within a single data center within a single Availability Zone (USE1-AZ4) in the US-EAST-1 Region. This is affecting availability and connectivity to EC2 instances that are part of the affected data center within the affected Availability Zone. We are also experiencing elevated RunInstance API error rates for launches wi…
This is quite interesting as they claim their datacenter design does better than Uptime's Tier3+ design requirements which require redundant power supply paths. [ https://aws.amazon.com/compliance/uptimeinstitute/ ]. I really hope they publish a thorough RCA for this incident.
So they have 2 different sources of power coming in. And generators. They do mention the UPS is only for "certain functions", so I guess it's not enough to handle full load while generators spin up if the 2 primaries go out. Or perhaps some failure in the source switching equipment (typically called a "static transfer switch").
Some detail on different approaches: https://www.donwil.com/wp-content/uploads/white-papers/Using...
The prevailing wisdom throughout the last couple of years was: “ditch your on-prem infrastructure and migrate to a major cloud provider” And its starting to seem like it could be something like: “ditch your on-prem infrastructure and spin up your own managed cloud” This is probably untenable for larger orgs where convenience gets the blank check treatment, but for smaller operations that can’t realize that value at s…
Self-managed infrastructure doesn’t fail now?
Commenters will show up like clockwork and say shit like:
“What man, it’s not like cars didn’t crash before? Haha”
Don’t be dense dude. And definitely don’t pursue a leadership position anytime in the future.
If you haven't seen yet, news is it was a power loss: > 5:01 AM PST We can confirm a loss of power within a single data center within a single Availability Zone (USE1-AZ4) in the US-EAST-1 Region. This is affecting availability and connectivity to EC2 instances that are part of the affected data center within the affected Availability Zone. We are also experiencing elevated RunInstance API error rates for launches wi…
How come they dont have power backups?
I'm not sure why this is a big deal though, this is why Amazon has multiple AZ's. If your in one AZ, you take your chances.
If you haven't seen yet, news is it was a power loss: > 5:01 AM PST We can confirm a loss of power within a single data center within a single Availability Zone (USE1-AZ4) in the US-EAST-1 Region. This is affecting availability and connectivity to EC2 instances that are part of the affected data center within the affected Availability Zone. We are also experiencing elevated RunInstance API error rates for launches wi…
How come they dont have power backups?
Earlier quoted context omitted.
Satire is the loophole of Copyright. If you satire ANYTHING you can use their copyrights in the satire. One could safely and legally drive an entire nation's transportation industry thru that loophole.
No, fair use does NOT allow you to retain the original copyright. That would be passing off a derived work as the original copyright holder's work, which could be very damaging. This is a violation of fair use, if it could even be considered that to begin with.
Wonder if it's connected?
Earlier quoted context omitted.
I should blog about this one day but... I have a server at OVH (not affiliated to them) which, at this point, I keep only for fun. It has 3162 days of uptime as I type this. 3 162 days. That's 8 years+ of uptime. Does it have the traffic of Amazon? No. Is it secure? Very likely not: it's running an old Debian version (Debian 7, which came out in, well, 2013). It only has one port opened though, SSH. And with quite a…
> Like that one time I solve LCS35 by computing for about four years on commodity hardware at home. Awesome! Are you Bernard Fabrot [0]? [0] https://www.csail.mit.edu/news/programmers-solve-mits-20-yea...