Live data from Hacker News

Facebook, Instagram, and WhatsApp outages

independent.co.uk

141–150 of 254 posts

Re: Facebook, Instagram, and WhatsApp outages

#141

Imagine if this meant not being able to send or receive currency.

You don't have to imagine - Wells Fargo was down for several hours earlier this year and no one could send or receive any money. People couldn't pay bills, transfer money and some couldn't even buy gas.

> some couldn't even buy gas.

How culture changes perspectives! Over here in Europe I was thinking "doesn't sound so bad". Then I realised over in the US, gas means petrol, and without petrol America doesn't work.

I'll reply to any replies after I've taken the electric train home.

Re: Facebook, Instagram, and WhatsApp outages

#142
post #135

Earlier quoted context omitted.

HN still seems to be working, perhaps they should consider hosting their own cloud?

HN is probably running on a LispMachine next to pg's desk. I wanted to say "under his desk", but you know, LispMachines...

Seems its still a single server. I think PG is off on sabbatical still though. https://news.ycombinator.com/item?id=18184749

Re: Facebook, Instagram, and WhatsApp outages

#143
post #107

Some popular images seem to load for me, but longer tail stuff doesn't. Maybe they accidentally flushed their edge cache and then their origin couldn't handle the load.

Could they be in your own browser's cache? Or maybe in your ISP's cache?

Considering the type of content Facebook, WhatsApp, and Instagram host, I'd imagine the cache hit rates to be pretty low for media. They'd be effective for JavaScript/CSS though, which might also be served from the same system

Re: Facebook, Instagram, and WhatsApp outages

#144
post #49

Looks like Facebook image store is having issues. Facebook, Instagram and WhatsApp use the same image store, so all the services are affected.

That’s also what it looks like to me. The services themselves work and are responsive here. But oh boy, images have been down for the longest time now for being top websites.

Re: Facebook, Instagram, and WhatsApp outages

#145
post #22
post #18

One interesting thing I've noticed from this (not sure if this is just my experience or if others noticed as well) but none of the ads seem to be impacted by this. Certainly makes me wonder if those are on a more prioritized and entirely separate infra and SLA that is designed to be more resilient and highly available.

It's a lot easier to serve ads than your custom photos. The ads are more generic, easier to cache globally, and show to a lot of people. They can also fall back to super generic ads if the database is unavailable to customize them. But your photos don't have a fallback.

Fallback is the main reason. Some webservices even fallback to non-moneymaking ads for charities in the case of a technical fault, because shareholders react badly to an ads outage, but they don't notice an hour or two of charity ads...

Re: Facebook, Instagram, and WhatsApp outages

#146
post #6

I find the outages of the last few days a curious coincidence : Slack, Google, Cloudflare, Azure, now FB. edit : added Slack after @kache_ mentioned it, added Azure after stevehubertron mentioned it.

Several european banks are also having issues with their login sites and online services. Backends are working though. I know, it's far fetched to throw them in the same bucket, and I won't, but it feels like there is happening some kind of reorganization on the net, which in the worst case would be some kind of global state-sponsored MITM-layer getting set up, maybe to prepare for new techs like 5G and TLS1.3.

My metaphorical tinfoil hat tells me that all the sudden security meetings yesterday (Putin, European Security Council, US VP Pence) were about this.

Re: Facebook, Instagram, and WhatsApp outages

#148

Earlier quoted context omitted.

Parent is referring to the holiday tomorrow in the United States.

I think GP understood quite well and wanted to express their dissatisfaction with the assumption that everyone is American.

Is there an official HN stance on this? There have been times I comment, with frustration, that a statement/positions can be hard to grok for those who don't have an automatically-American context in mind (you know, because they might not live in America) the downvotes come swiftly and snarky comments about "this is an American messageboard" follow.

So I'm actually genuinely curious what YC/HN's official position on this is. Asking broadly.

Re: Facebook, Instagram, and WhatsApp outages

#149
post #136

Do you remember when we first heard about "the cloud"? I remind about a video when Larry Ellison mentioned the cloud and I thought: "What is he speaking about? It's the internet". How I was wrong... Now we are in a situation where we have too much concentration and we are experiencing that. It's a problem.

I wonder if there's a large enough market for a "decentralized cloud" along the lines of IPFS but for compute. And, yknow, actually working decently. Something where you could pay compute/storage operators for resources and also be a server host and share your excess compute power.

This is effectively the idea behind the Ethereum blockchain model. You can make "smart contracts" (think APIs with persistent state) using languages like Solidity[1] and deploy them onto the blockchain. After that, you can invoke individual functions by paying "gas" (small amounts of ETH) that goes to the node operator's account. Smart contracts also get their own blockchain addresses, so they're capable of sending and receiving transactions. Meaning you can build financial applications with little to no barriers to entry! (whether this is a good thing or not, I will leave up to the reader).

The really cool thing about it is that nobody owns a smart contract once it's deployed. You can't edit the code or even delete the contract itself. It's a truly autonomous entity that will continue to operate the same way forever (unless there's a 51% attack or something of that nature).

The obvious benefit for this is that you can mathematically ensure trust. For example, if you hosted a lottery app on a LAMP stack, you could steal the money, hackers could get into your server, your database could get corrupted, etc. On the blockchain, nobody can access your lottery funds or business logic, not even yourself, meaning that as long as you developed the application correctly (to be fair, that is a big assumption), it is truly fair.

[1] https://solidity.readthedocs.io/en/v0.5.10/

Re: Facebook, Instagram, and WhatsApp outages

#150
post #107

Some popular images seem to load for me, but longer tail stuff doesn't. Maybe they accidentally flushed their edge cache and then their origin couldn't handle the load.

Could they be in your own browser's cache? Or maybe in your ISP's cache? Considering the type of content Facebook, WhatsApp, and Instagram host, I'd imagine the cache hit rates to be pretty low for media. They'd be effective for JavaScript/CSS though, which might also be served from the same system

No, I checked if it was my own cache, and it's all HTTPS so it couldn't have been my ISP's cache either.

I compared a few celebrity's instagrams with accounts of friends with many fewer followers. Celeb's images loaded but less and less as you scroll back through time, friends images all 503'd. Still seems to be the case.

Post reply on HN