Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fastly Outage

fastly.com

Post reply on HN
Print view View original

461–470 of 740 posts

Re: Fastly Outage

#461

Post by ImpactStrafe »

evouga wrote: post #373
ImpactStrafe wrote: post #337

Earlier quoted context omitted.

Fastly is a Content Distribution Network (CDN). Basically the closer the server serving the webpage is to the end user the faster it is for the end user to see and interact with. But running servers all over the world 1) isn't efficient 2) costs a lot of money. So a few companies (fastly, cloud flare, akamai) figured, hey, why don't we build a bunch of small data centers all over the world and then provide a distribu…

Thanks. That makes sense. Wouldn’t you build in a failsafe that bypasses Fastly and sends traffic to your own servers in the case of this kind of outage? Or outages are so rare that it’s not worth the trouble?

The number of serious CDN outages in the world are incredibly rare.

In fact, you can probably remember most of them if you were given dates.

Plus, going around the CDN can be very complex (depending on the type of content), very expensive (all of a sudden you have a massive data out network traffic that didn't exist previously), and not guaranteed to work (DNS updates can take longer to get to everyone than the actual CDN outage lasts).

There are places where it is worth it and useful, but for a lot of the sites listed it's not useful.

Re: Fastly Outage

#462

Post by Dumbdo »

evouga wrote: post #373
ImpactStrafe wrote: post #337

Earlier quoted context omitted.

Fastly is a Content Distribution Network (CDN). Basically the closer the server serving the webpage is to the end user the faster it is for the end user to see and interact with. But running servers all over the world 1) isn't efficient 2) costs a lot of money. So a few companies (fastly, cloud flare, akamai) figured, hey, why don't we build a bunch of small data centers all over the world and then provide a distribu…

Thanks. That makes sense. Wouldn’t you build in a failsafe that bypasses Fastly and sends traffic to your own servers in the case of this kind of outage? Or outages are so rare that it’s not worth the trouble?

> Or outages are so rare that it’s not worth the trouble?

This, I can't remember the last Fastly outage in this dimension, so the time spent on setting up a secondary server serving your assets is probably not really worth it for small-medium companies. Although i'd think otherwise for a company like Shopify.

Re: Fastly Outage

#463

Post by optiomal_isgood »

k_ wrote: post #412

Update: The issue has been identified and a fix is being implemented. Posted Jun 08, 2021 - 10:44 UTC Seems like this is being resolved; curious to see the details afterwards (from https://status.fastly.com/incidents/vpk0ssybt3bj )

Reddit, Stack Overflow, Spotify, all back for me. Good job Fastly engineers!

Re: Fastly Outage

#465

Post by danpalmer »

kypro wrote: post #390
barosl wrote: post #53

I didn't know so many sites were depending on Fastly. Stack Overflow, GitHub, reddit, .... Even pip is unavailable. My development workflow is completely janked up. It is a bit scary that we are putting too many eggs in one basket.

You would think sites like Github and key government sites would at least have a fall back at the ready. It reasonable to use a CDN like Fastly, but having a single point of failure seems silly if you're the BBC or Gov UK. Although, it does seem BBC managed to get back up and running pretty quick so perhaps they were prepared for this.

Gov.UK is back up too. They have a mandate from government to be able to provide emergency communications so I expect they did have a backup and have managed to switch over, but just took 30 mins to do so.

Gov.UK is supposed to be a bit like BBC 1 or Radio 1 – in a national emergency they can be taken over to disseminate critical information, like if there was a nuclear attack launched on the UK.

Re: Fastly Outage

#467

Post by patentatt »

0xbkt wrote: post #118
csomar wrote: post #62

So I'm wondering where in the "hundreds of servers around the world" did they exactly go wrong. This happened with Cloudflare before too. I think we are a little too dependent on these services.

It is a meaningless premise when you actually have SPoFs baked deep inside the system.

I’d love to see a breakdown of what single point of failure causes these worldwide network outages. They even brag about redundancy in their marketing materials. I hope we see a post mortem on this

Re: Fastly Outage

#468

Post by s_fischer »

xuki wrote: post #357
unityByFreedom wrote: post #309

Earlier quoted context omitted.

While we're here.. I am a bit surprised to see how many sites use Fastly. As a dev I've always been happy with Cloudflare.

Now imagine how many sites would go down if it was CF

No need to imagine! Just search HN for "cloudflare outage" and you'll see that it happened several times over the last few years

Re: Fastly Outage

#469

Post by [deleted] »

collyw wrote: post #383
lpmitchell wrote: post #9

This seems to be impacting a number of huge sites, including the UK government website[0]. [0] https://www.gov.uk/ https://m.media-amazon.com/ https://pages.github.com/ https://www.paypal.com/ https://stackoverflow.com/ https://nytimes.com/ Edit: Fastly's incident report status page: https://status.fastly.com/incidents/vpk0ssybt3bj

Click the new tab. Lots of posts about sites being down. All flagged.

[deleted]

Re: Fastly Outage

#470

Post by 00deadbeef »

optiomal_isgood wrote: post #216

Amazon.com was completely broken here (Europe) and they're back, I was observing from where the assets were loaded from and they switched from EU to NA as a failover. Homework well done.

I was surprised to learn Amazon don't use their own CDN
Post reply on HN