Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fastly Outage

fastly.com

Post reply on HN
Print view View original

411–420 of 740 posts

Re: Fastly Outage

#411

Post by cowsandmilk »

lpmitchell wrote: post #108
schappim wrote: post #81

Earlier quoted context omitted.

Yeah, this is what makes me feel this is more an AWS thing

The m.media-amazon.com domain (and a few other CDN'd domains that they use) are running through Fastly: nslookup m.media-amazon.com Name: media.amazon.map.fastly.net It is very interesting that they are not using CloudFront!

really, m.media-amazon.com seems to have a very short TTL (showing 37 seconds right now) and has been weighted to cloudfront now.

Amazon is also known to use Akamai. Sure, Amazon relies heavily on AWS, but why should it surprise anyone that a retail website obsessed with instant loading of pages decides to use non-AWS CDNs if the performance is better.

Even if CloudFront became the default, I'm certain amazon.com would keep contracts with fastly and akamai just so they can weight traffic away from CloudFront in an outage.

Re: Fastly Outage

#413

Post by hardwaresofton »

mothsonasloth wrote: post #333
austinjp wrote: post #166

Yeah so it's been mentioned in the comments already, but to everyone in Fastly right now: I feel for you. Something like this must be insanely stressful, and not just during the outage. There will be (should be) a massive post-mortem. People will be losing sleep over this for days, weeks, months. :( Edit: There seems to be a major empathy outage in this thread. Disgusted but not surprised, unfortunately.

Call me old fashioned but the latest trend of showing "empathy" for a serious incident, then proceeding to dance around the aftermath of it, whilst people give themselves a pat on back in a retro/post-mortem, isn't the way to do it. People need to be blamed, and responsibility for actions taken (without covering asses)

I hear you, but I just want to point out that this rarely happens anywhere else. It's great if tech (and people in general) hold themselves to progressively higher standards than what is out there already, but I don't think tech needs to be that much better, I'd settle for just doing a good honest retro (without throwing anyone under the bus, and without covering their asses)

A good leader will take the hit (and the repercussions) for their underlings, compensate customers where compensation can make it better (and offer to make it easy to use fallbacks if this happens again) -- and internally fix the problem so it can't happen again, without throwing anyone to the dogs.

Re: Fastly Outage

#414

Post by bartread »

magicturtle wrote: post #49
lpmitchell wrote: post #9

This seems to be impacting a number of huge sites, including the UK government website[0]. [0] https://www.gov.uk/ https://m.media-amazon.com/ https://pages.github.com/ https://www.paypal.com/ https://stackoverflow.com/ https://nytimes.com/ Edit: Fastly's incident report status page: https://status.fastly.com/incidents/vpk0ssybt3bj

reddit down aswell

It's OK though, because large swathes of this discussion seem to have turned HN into reddit, at least temporarily. Normal service will no doubt resume in due course.

Re: Fastly Outage

#415

Post by sergiomattei »

TheRealDunkirk wrote: post #387

Every other comment about what's down in this thread -- as if we needed dozens of site-by-site accountings of this outage in the first place -- is a bitch about reddit. Why is reddit so important to this crowd? The specific topics I used to read the site for (half a dozen years ago) have all been overrun by "bucket people," there is literally never an answer to any question I find a google link to there, and the site…

Of course, the Enlightened Folk of this site can no longer use their leisure time on lowly activities such as the "Reddit".

Teach me your ways, master! /s

Jokes aside, people can do whatever they please. Reddit has a bunch of niche communities around many hobbies and fun things. No need to be bitter about it.

Re: Fastly Outage

#416

Post by itsbits »

ClearAndPresent wrote: post #71

What conclusions can we draw about concentrating web content in a few CDNs?

we had that experience when cloudfare was down for sometime lastyear. We now setup a minor own static server as a backup, if at all this happens again. Althgh we hadn't so far had to use it.

Re: Fastly Outage

#417

Post by OJFord »

mothsonasloth wrote: post #333
austinjp wrote: post #166

Yeah so it's been mentioned in the comments already, but to everyone in Fastly right now: I feel for you. Something like this must be insanely stressful, and not just during the outage. There will be (should be) a massive post-mortem. People will be losing sleep over this for days, weeks, months. :( Edit: There seems to be a major empathy outage in this thread. Disgusted but not surprised, unfortunately.

Call me old fashioned but the latest trend of showing "empathy" for a serious incident, then proceeding to dance around the aftermath of it, whilst people give themselves a pat on back in a retro/post-mortem, isn't the way to do it. People need to be blamed, and responsibility for actions taken (without covering asses)

I don't agree about the blame, but I do also find the empathy cringeworthy. Something's broken; someone's job is to fix it; they'll fix it; it will work again. /shrug/

Post-mortems make far more interesting submissions IMO, but I suppose people up-vote 'yes down for me too'.

Re: Fastly Outage

#418

Post by perryizgr8 »

TheDauthi wrote: post #191
sdflhasjd wrote: post #85

Earlier quoted context omitted.

I fully expect that if I find a "major outage" on Slack's status page that it could only mean the outbreak of nuclear war.

"Some users may experience brief service disruption."

"Only the last couple seconds of their lives."

Re: Fastly Outage

#419

Post by darkcha0s »

mothsonasloth wrote: post #333
austinjp wrote: post #166

Yeah so it's been mentioned in the comments already, but to everyone in Fastly right now: I feel for you. Something like this must be insanely stressful, and not just during the outage. There will be (should be) a massive post-mortem. People will be losing sleep over this for days, weeks, months. :( Edit: There seems to be a major empathy outage in this thread. Disgusted but not surprised, unfortunately.

Call me old fashioned but the latest trend of showing "empathy" for a serious incident, then proceeding to dance around the aftermath of it, whilst people give themselves a pat on back in a retro/post-mortem, isn't the way to do it. People need to be blamed, and responsibility for actions taken (without covering asses)

I'm sure you've never made a mistake.

The best way (in a team), to tackle mistakes, is to ensure the process in place corrects these mistakes. The only way to do that, is a post-mortem/learning from the mistake. If you blame it on some engineer who did it, that guy will eventually be replaced by some other guy, who may make the same mistake.

Re: Fastly Outage

#420

Post by busymom0 »

strictfp wrote: post #315
austinjp wrote: post #166

Yeah so it's been mentioned in the comments already, but to everyone in Fastly right now: I feel for you. Something like this must be insanely stressful, and not just during the outage. There will be (should be) a massive post-mortem. People will be losing sleep over this for days, weeks, months. :( Edit: There seems to be a major empathy outage in this thread. Disgusted but not surprised, unfortunately.

Meh. Losing sleep sounds like an over-reaction. No system is foolproof. Of course Fastly should do what they can to prevent downtime, but it's still expected that they will go down. I would blame anyone who claimed otherwise or couldn't deal with it while not having a fallback.

I think the government websites being down (UK ones for example) are the bigger issue. Reddit/Stackoverflow etc being down isn't that big of a deal imo.
Post reply on HN