351–360 of 544 posts
Earlier quoted context omitted.
How does the fact you worked at those companies relate to #4? Edit: I misread the parent and my question doesn't make a lot of sense. Please ignore it :)
> How does the fact you worked at those companies relate to #4? For Facebook I worked on the incident, previous Wednesday. 9.5 hours of pain... And for my past employers, I still have friends there texting the root causes with facepalm emojis.
So storytime! I worked at Twitter as a contractor in 2008 (my job was to make internal hockey-stick graphs of usage to impress investors) during the Fail Whale era. The site would go down pretty much daily, and every time the ops team brought it back up, Twitter's VCs would send over a few bottles of really fancy imported Belgian beer (the kind with elaborate wire bottle caps that tell you it's expensive). I would in…
What really made me mad was when we hired consultants and the contract would end, usually without much success because Twitter's problems were not normal problems, and then they would send us a fancy gift basket with our own wasted money.
Maciej, we are still waiting for you to ship the executive dashboard.
Ok, this is too many high-profile, apparently unrelated outages in the last month to be completely a coincidence. Hypotheses: 1) software complexity is escalating over time, and logically will continue to until something makes it stop. It has now reached the point where even large companies cannot maintain high reliability. 2) internet volume is continually increasing over time, and periodically we hit a point where…
One possibility on 5) Too many KPIs and quarter goals to be reached, too many corners cut.
"Let's deploy to production" https://youtu.be/5p8wTOr8AbU
Ok, this is too many high-profile, apparently unrelated outages in the last month to be completely a coincidence. Hypotheses: 1) software complexity is escalating over time, and logically will continue to until something makes it stop. It has now reached the point where even large companies cannot maintain high reliability. 2) internet volume is continually increasing over time, and periodically we hit a point where…
6) White House social media conference just started. https://www.10tv.com/article/trump-hosts-white-house-summit-...
mastodon.social is also suddenly down.
No, it's not
Earlier quoted context omitted.
> I would intercept these rewards and put them in my backpack for the bus ride home, in order to avoid creating perverse incentives for the operations team. But did anyone call me 'hero'? Wait, I don't understand. Why would anyone call you hero? Are you suggesting that the team would deliberately crash the app to receive beers and that by stealing them you stopped this from happening? Free drinks and free food is the…
> Am I missing something? That it is a joke.
Really curious which part of their infrastructure was the root cause.
source: work there