Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Twitter was down

status.twitterstat.us

Post reply on HN
Print view View original

451–460 of 544 posts

Re: Twitter was down

#451

Post by doomjunky »

whatshisface wrote: post #69

I remember once we were at three outages, someone posted that they thought three was a reasonably-sized random cluster given the rate at which services go down. How many outages have we had in the last 30 days, how many do we have per month on average, and how strongly can we reject the null hypothesis? The formula for computing how unlikely this is is the Poisson distribution: `λ^k * e^-λ / k!`, where λ is the avera…

07/11/2019 Twitter outage

07/03/2019 WhatsApp, Facebook & Instagram outage

07/02/2019 Cloudflare outage (Discord, 9gag, Medium)

07/02/2019 Google Cloud glass fiber damage (Google Services)

06/24/2019 Verizon route leak (Cloudflare, Google, Amazon, Reddit)

06/02/2019 Google Cloud excalated outage progression (G Suite, YouTube)

Re: Twitter was down

#452

Post by idlewords »

bdd wrote: post #279
rossdavidh wrote: post #100

Ok, this is too many high-profile, apparently unrelated outages in the last month to be completely a coincidence. Hypotheses: 1) software complexity is escalating over time, and logically will continue to until something makes it stop. It has now reached the point where even large companies cannot maintain high reliability. 2) internet volume is continually increasing over time, and periodically we hit a point where…

#4 I work at Facebook. I worked at Twitter. I worked at CloudFlare. The answer is nothing other than #4. #1 has the right premise but the wrong conclusion. Software complexity will continue escalating until it drops by either commoditization or redefining problems. Companies at the scale of FAANG(+T) continually accumulate tech debt in pockets and they eventually become the biggest threats to availability. Not the ne…

FAANG(+T)(-N)(+M)

Re: Twitter was down

#453

Post by wybiral »

bdd wrote: post #279
rossdavidh wrote: post #100

Ok, this is too many high-profile, apparently unrelated outages in the last month to be completely a coincidence. Hypotheses: 1) software complexity is escalating over time, and logically will continue to until something makes it stop. It has now reached the point where even large companies cannot maintain high reliability. 2) internet volume is continually increasing over time, and periodically we hit a point where…

#4 I work at Facebook. I worked at Twitter. I worked at CloudFlare. The answer is nothing other than #4. #1 has the right premise but the wrong conclusion. Software complexity will continue escalating until it drops by either commoditization or redefining problems. Companies at the scale of FAANG(+T) continually accumulate tech debt in pockets and they eventually become the biggest threats to availability. Not the ne…

I've still never seen this much downtime on these systems so it's weird to happen all at once.

It's possible that they're related without requiring any conspiracy theories or anything. Maybe these companies are just getting too big or too sloppy to maintain the same standard of uptime (compared to the past few years)? Or maybe there's some underlying issue that they're all rushing to fix which justifies the breaking prod changes within the same timeframe.

But it was weird when a it happened to two or three of them. Now we're going on something like 5 massive failures from some of the biggest services online within a little over a week...

Re: Twitter was down

#454

Post by edflsafoiewq »

thfuran wrote: post #412
t0astbread wrote: post #386

Earlier quoted context omitted.

Is it not? Sorry if I got it wrong, English isn't my first language. dict.cc (my source of truth for English vocab) says it's a word: https://www.dict.cc/?s=tendentially

It's apparently a word but I'd say it's quite uncommon. I played around with google ngram viewer and had a hard time coming up with a word that is less common. But I finally came up with "astrophotographic". E: "unsurpassingly" is way down there too

It's very common in Biblical criticism (transliterated from German).

Re: Twitter was down

#455

Post by dang »

loblollyboy wrote: post #378
bdd wrote: post #368

Earlier quoted context omitted.

geez, tough crowd. do you wanna ten dollar hug?

I was just polishing my bit. Not in a bad mood today so much as a bored mood. You seem like you know what you are talking about (yes, I was bored enough to stalk you, too)

Could you please stop posting unsubstantive comments to Hacker News?

Re: Twitter was down

#456

Post by KuhlMensch »

Years ago I read an amazing article (from HN) about how (complex) config, rather than code ends up being the cause of outages at scale. I always reflect on that, when designing almost anything these days.

Re: Twitter was down

#457

Post by pard68 »

rossdavidh wrote: post #100

Ok, this is too many high-profile, apparently unrelated outages in the last month to be completely a coincidence. Hypotheses: 1) software complexity is escalating over time, and logically will continue to until something makes it stop. It has now reached the point where even large companies cannot maintain high reliability. 2) internet volume is continually increasing over time, and periodically we hit a point where…

NSA firmware updates requiring a reboot.

Re: Twitter was down

#458

Post by iamtheworstdev »

bdd wrote: post #279
rossdavidh wrote: post #100

Ok, this is too many high-profile, apparently unrelated outages in the last month to be completely a coincidence. Hypotheses: 1) software complexity is escalating over time, and logically will continue to until something makes it stop. It has now reached the point where even large companies cannot maintain high reliability. 2) internet volume is continually increasing over time, and periodically we hit a point where…

#4 I work at Facebook. I worked at Twitter. I worked at CloudFlare. The answer is nothing other than #4. #1 has the right premise but the wrong conclusion. Software complexity will continue escalating until it drops by either commoditization or redefining problems. Companies at the scale of FAANG(+T) continually accumulate tech debt in pockets and they eventually become the biggest threats to availability. Not the ne…

Faangt = Facebook amazon Apple Netflix Google tesla?

Re: Twitter was down

#459

Post by euske »

newsbinator wrote: post #424
mastratton3 wrote: post #411

Earlier quoted context omitted.

lol yes, whats the quote on "Don't assume bad intention when incompetence is to blame"? After seeing how people write code in the real world, I'm actually surprised there aren't more outages.

Hanlon's Razor: https://en.wikipedia.org/wiki/Hanlon%27s_razor "Never attribute to malice that which is adequately explained by stupidity."

I always thought that this cause should also include "greed". But then, greed is kinda one step closer to malice, and I'm not sure if there's a line.

Re: Twitter was down

#460

Post by jacobgreenleaf »

rococode wrote: post #113
lukey_q wrote: post #25

A lot of high-profile outages recently. Can't actually remember the last time Twitter went fully down. Have to confess I immediately assumed an issue with my own connection, even though every other site is working. Unrelated, but for some reason the phrase "I have no mouth and I must scream" just popped into my head

Twitter is especially weird for this since it's often a platform where people talk about downtimes. I don't see this downtime mentioned on Reddit and I don't know of other sites where it might be discussed, so if Hacker News happened to go down at the same time, where would I go to talk about it with online strangers and find out if it's just me? Nowhere, I guess, I'd just wait it out with no extra insights on what's…

You could use the outages mailing list

https://puck.nether.net/mailman/listinfo/outages

https://puck.nether.net/pipermail/outages/2019-July/012527.h...

Post reply on HN