Fastly Outage • Hacker News Board

Re: Fastly Outage

#111

Post by lysp » Tue, Jun 08, 2021, 10:11 AM UTC

nineteen999 wrote: post #101

Haydos585x2 wrote: post #59

Such a huge number of sites. It seems like it's mostly US based sites and Australians are okay. Sending good vibes to whatever poor person is on support right now.

I'm in Australia and there are heaps of sites down for me.

As per report above - most (or all?) of Asia/Pac servers are down.

This incident affects: North America (Ashburn (BWI), Ashburn (DCA), Ashburn (IAD)), Europe (Amsterdam (AMS)), and Asia/Pacific (Hong Kong (HKG), Tokyo (TYO), Singapore (QPG)).

Re: Fastly Outage

#112

Post by dspillett » Tue, Jun 08, 2021, 10:11 AM UTC

threeseed wrote: post #45

Shopify's CDN is down. Which is causing $15+ million in lost product sales for every hour of outage. Not to mention the loss of any new customers.

StackOverflow and all the StackExchange family of sites are down. I suspect the lost productivity from that will be more costly over the whole economy than potential lost sales via Shopify. People can go back to shopify so those transactions not definitely blocked for ever, any time "lost" due to reference resources being unavailable can't so easily be claimed back.

Re: Fastly Outage

#113

Post by threeseed » Tue, Jun 08, 2021, 10:11 AM UTC

schappim wrote: post #81

threeseed wrote: post #68

Earlier quoted context omitted.

I wonder why Amazon is not using Cloudfront for their own website.

Yeah, this is what makes me feel this is more an AWS thing

AWS is reporting no issues across the regions:

https://status.aws.amazon.com

Re: Fastly Outage

#114

Post by iso1631 » Tue, Jun 08, 2021, 10:11 AM UTC

adamjb wrote: post #93

JrProgrammer wrote: post #76

Earlier quoted context omitted.

> This incident affects: North America (Ashburn (BWI), Ashburn (DCA)). How come we are affected by this in the Netherlands?

They've updated it to >North America (Ashburn (BWI), Ashburn (DCA), Ashburn (IAD)), Europe (Amsterdam (AMS)), and Asia/Pacific (Hong Kong (HKG), Tokyo (TYO), Singapore (QPG)).

I've seen errors return to me referencing a LON (London I assume) server - Details: cache-lon4238-LON for example

Re: Fastly Outage

#115

Post by benrbray » Tue, Jun 08, 2021, 10:12 AM UTC

lpmitchell wrote: post #9

This seems to be impacting a number of huge sites, including the UK government website[0]. [0] https://www.gov.uk/ https://m.media-amazon.com/ https://pages.github.com/ https://www.paypal.com/ https://stackoverflow.com/ https://nytimes.com/ Edit: Fastly's incident report status page: https://status.fastly.com/incidents/vpk0ssybt3bj

https://hackage.haskell.org/ down as well

Re: Fastly Outage

#116

Post by weird-eye-issue » Tue, Jun 08, 2021, 10:12 AM UTC

superzamp wrote: post #61

algo_cheese wrote: post #19

Earlier quoted context omitted.

> We're currently investigating potential impact to performance with our CDN services. Guys, you are offline with a 503 error, this is a little more than "potential impact to performance".

Lowkey status reports are the norm now :) "some users may experience degraded service" => site completely down for all locations

Well to be fair some users were not accessing the site at that time

Re: Fastly Outage

#117

Post by secondcoming » Tue, Jun 08, 2021, 10:12 AM UTC

creamyhorror wrote: post #12

basically the internet is down reddit, stackoverflow, github, paypal, pypi, twitter, twitch, NYT, CNN, BBC, the Guardian... edit: wow, even Amazon.com relies on Fastly for some of its edge caches!

BBC is still up at least in the UK

Re: Fastly Outage

#118

Post by 0xbkt » Tue, Jun 08, 2021, 10:12 AM UTC

csomar wrote: post #62

So I'm wondering where in the "hundreds of servers around the world" did they exactly go wrong. This happened with Cloudflare before too. I think we are a little too dependent on these services.

It is a meaningless premise when you actually have SPoFs baked deep inside the system.

Re: Fastly Outage

#119

Post by rvz » Tue, Jun 08, 2021, 10:12 AM UTC

lpmitchell wrote: post #9

This seems to be impacting a number of huge sites, including the UK government website[0]. [0] https://www.gov.uk/ https://m.media-amazon.com/ https://pages.github.com/ https://www.paypal.com/ https://stackoverflow.com/ https://nytimes.com/ Edit: Fastly's incident report status page: https://status.fastly.com/incidents/vpk0ssybt3bj

Centralising everything™ and the whole internet goes down because of that.

Re: Fastly Outage

#120

Post by [deleted] » Tue, Jun 08, 2021, 10:12 AM UTC

lpmitchell wrote: post #9

This seems to be impacting a number of huge sites, including the UK government website[0]. [0] https://www.gov.uk/ https://m.media-amazon.com/ https://pages.github.com/ https://www.paypal.com/ https://stackoverflow.com/ https://nytimes.com/ Edit: Fastly's incident report status page: https://status.fastly.com/incidents/vpk0ssybt3bj

[deleted]