isitdownrightnow.com is down
Thanks for the best laughs in a while friend - that's pure irony right there!
Fastly Outage
711–720 of 740 posts
Re: Fastly Outage
#712This post is suspiciously ranked much lower than it should be (1216 points, 9 hours ago), lower than posts with < 100 points.
Re: Fastly Outage
#713Amazon.com was completely broken here (Europe) and they're back, I was observing from where the assets were loaded from and they switched from EU to NA as a failover. Homework well done.
I was surprised to learn Amazon don't use their own CDN
Amazon (like a lot of others) use several CDNs for redundancy. You can see from dig that it resolves to combinations of cloudfront, akamai, and (presumably, based on your reported experience) fastly.
dig +short www.amazon.com
tp.47cf2c8c9-frontier.amazon.com.
d3ag4hukkh62yn.cloudfront.net.
65.8.70.16
dig +short www.amazon.co.uk
tp.bfbdc3ca1-frontier.amazon.co.uk.
dmv2chczz9u6u.cloudfront.net.
13.224.0.89
dig +short www.amazon.in
tp.c95e7e602-frontier.amazon.in.
d1elgm1ww0d6wo.cloudfront.net.
13.224.9.30
dig +short www.amazon.co.jp
tp.4d5ad1d2b-frontier.amazon.co.jp.
www.amazon.co.jp.edgekey.net.
e15312.a.akamaiedge.net.
104.71.134.162Re: Fastly Outage
#714Every other comment about what's down in this thread -- as if we needed dozens of site-by-site accountings of this outage in the first place -- is a bitch about reddit. Why is reddit so important to this crowd? The specific topics I used to read the site for (half a dozen years ago) have all been overrun by "bucket people," there is literally never an answer to any question I find a google link to there, and the site…
reddit taught me to never trust a mod, so it does have some purpose still. i think without glaringly bad examples of how (not) to run a community based site, we would be doomed to repeat it's mistakes.
Re: Fastly Outage
#715Earlier quoted context omitted.
Reddit's attempts at dark patterns are embarrassing from all perspectives. If you use dark patterns it's a laughably abysmal implementation. If you abhor dark patterns, it's a frustration.
It's just enough to annoy you but not enough to make everyone leave the platform
I honestly thought Reddit would die when they introduced Reddit awards, it seemed like such an obvious cash grab. You can't underestimate the amount of community momentum that the site has though.
Re: Fastly Outage
#716Shopify's CDN is down. Which is causing $15+ million in lost product sales for every hour of outage. Not to mention the loss of any new customers.
[deleted]
Re: Fastly Outage
#717Earlier quoted context omitted.
I disagree. People implemented those systems, so if you are correct that it is the systems fault, then it is also a persons fault. People must be held accountable to have good incentives to reduce such outtages in the future. I do agree though that we should always be compassionate and realistic with other humans.
> I disagree. People implemented those systems, so if you are correct that it is the systems fault, then it is also a persons fault. How do you make sure that mistakes don't happen, then? Do you blame and fire people who make mistakes, and hope that the next person put in the same spot doesn't make a mistake? Or do you figure out what caused that person to make the mistake and ensure there are processes in place so t…
I see: When I said we need to hold people accountable, you may have heard that we need to fire people. That was honestly never on my mind.
Maybe its a cultural thing. I hear a lot of firing at the US. I am from Europe.
Re: Fastly Outage
#718Earlier quoted context omitted.
I disagree. People implemented those systems, so if you are correct that it is the systems fault, then it is also a persons fault. People must be held accountable to have good incentives to reduce such outtages in the future. I do agree though that we should always be compassionate and realistic with other humans.
v1. "It's Bob's fault and so we fired Bob." v2. "The issue was caused by a previously unidentified pathway that caused a feedback loop and overloaded our servers in a cascading fashion (or whatever). We have implemented a fix for this and updated our testing and deployment processes to stop similar cascades." Which solves the problem long term? As an architect making product choices, v2 wins every time. (With the cav…
As I replied to a sibling comment, I never thought about firing Bob. I think we can assign responsibilities without being mean or denegrate someone.
I am critizing myself all the time for stuff. No hurt feelings there.
Re: Fastly Outage
#719Earlier quoted context omitted.
> pretty certain this is not how Netflix's culture is. > pReTtY CeRtAiN This, the wording in of itself shows you have absolutely no clue whatsoever at all of Netflix's culture.
Riiiight... Anyways, you kept complaining of being downvoted, here's a clue: you're being an ass and no one likes you or what you have to say because you're wrong. So go scurry back to reddit where you belong troll...
> you're being an ass and no one likes you or what you have to say because you're wrong. So go scurry back to reddit where you belong troll...
Okay? some proof please? This is not far off from a baseless character attack which isn't really effective when trying to convince me about your point on you knowing about Netflix's culture.
If you really want a proper answer, the truth is, unfortunately for you I am in management (previously was an engineer) and have always known Netflix to have a stellar performance oriented (and fear driven) culture, their playbook operates like a sports team. Not for everyone, but that's the point and it works for them.
Maybe you should look inward to yourself if you're so vexed with me to call me silly names, that you can't handle the truth or the culture about why some companies like Netflix adopts this.
Peace.
Re: Fastly Outage
#720Earlier quoted context omitted.
> What's your SLA with them? Sorry what? You've just witnessed almost the entire internet break because of a catastrophic cascading outage that affected lots of huge companies, since third party services used and trusted Fastly. Shopify stores couldn't accept payments on their websites, Coinbase Retail/Pro transactions and trading apps failed to load, and delivery apps stopped loading all of a sudden. These are just…
> You've just witnessed almost the entire internet break because of a catastrophic cascading outage that affected lots of huge companies, since third party services used and trusted Fastly. Blame the companies that relied on Fastly being up 100% of the time, even though Fastly explicitly states that they might be down any number of hours, and they will even give you money back for that [1]. If they did offer 100% SLA…
The main problem is that they had a failover system, the mystery is where was it in this outage?
Why didn't this trigger? where was this system in place to prevent further cascading failures?
> Blame the companies that relied on Fastly
So it's everybody's fault Fastly went down now? That is a new one.