Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reddit Releases Post Mortem for Its 3 Hour Outage Last Week

old.reddit.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Reddit Releases Post Mortem for Its 3 Hour Outage Last Week

#11

Post by Nextgrid »

londons_explore wrote: post #10
mwint wrote: post #5

> In the 1.20 series, Kubernetes changed its terminology from “master” to “control-plane.” And in 1.24, they removed references to “master,” even from running clusters. This is the cause of our outage. Kubernetes node labels. Wow, so the word police brought down Reddit. Why on earth did someone think it a good idea to screw with existing names in running clusters in a cluster management tool?

The word police seem to have gone kinda quiet lately. I wonder if they realised that banning a few words wasn't really helping their cause.

Those bullshit positions are the first to go in any kind of recession or downturn, that's the most likely explanation.

Re: Reddit Releases Post Mortem for Its 3 Hour Outage Last Week

#12

Post by [deleted] »

londons_explore wrote: post #10
mwint wrote: post #5

> In the 1.20 series, Kubernetes changed its terminology from “master” to “control-plane.” And in 1.24, they removed references to “master,” even from running clusters. This is the cause of our outage. Kubernetes node labels. Wow, so the word police brought down Reddit. Why on earth did someone think it a good idea to screw with existing names in running clusters in a cluster management tool?

The word police seem to have gone kinda quiet lately. I wonder if they realised that banning a few words wasn't really helping their cause.

[deleted]

Re: Reddit Releases Post Mortem for Its 3 Hour Outage Last Week

#14

Post by hoseja »

mwint wrote: post #5

> In the 1.20 series, Kubernetes changed its terminology from “master” to “control-plane.” And in 1.24, they removed references to “master,” even from running clusters. This is the cause of our outage. Kubernetes node labels. Wow, so the word police brought down Reddit. Why on earth did someone think it a good idea to screw with existing names in running clusters in a cluster management tool?

I laughed out loud when I got to that point. Well deserved.

Re: Reddit Releases Post Mortem for Its 3 Hour Outage Last Week

#15

Post by c7DJTLrn »

mwint wrote: post #5

> In the 1.20 series, Kubernetes changed its terminology from “master” to “control-plane.” And in 1.24, they removed references to “master,” even from running clusters. This is the cause of our outage. Kubernetes node labels. Wow, so the word police brought down Reddit. Why on earth did someone think it a good idea to screw with existing names in running clusters in a cluster management tool?

I'm sure it was worth making breaking changes to make Kubernetes more "inclusive"... whatever that means. As if Kubernetes ever excluded anybody, or "master" referred to slavery in any way.

Re: Reddit Releases Post Mortem for Its 3 Hour Outage Last Week

#17

Post by tapoxi »

c7DJTLrn wrote: post #15
mwint wrote: post #5

> In the 1.20 series, Kubernetes changed its terminology from “master” to “control-plane.” And in 1.24, they removed references to “master,” even from running clusters. This is the cause of our outage. Kubernetes node labels. Wow, so the word police brought down Reddit. Why on earth did someone think it a good idea to screw with existing names in running clusters in a cluster management tool?

I'm sure it was worth making breaking changes to make Kubernetes more "inclusive"... whatever that means. As if Kubernetes ever excluded anybody, or "master" referred to slavery in any way.

These terminology changes date back to the George Floyd protests, and instead of getting action to solve the actual problem, we got blog posts from GitHub about changing the default branch name, so they can feel like they're doing something and elevate their brand.

Re: Reddit Releases Post Mortem for Its 3 Hour Outage Last Week

#18

Post by londons_explore »

londons_explore wrote: post #10
mwint wrote: post #5

> In the 1.20 series, Kubernetes changed its terminology from “master” to “control-plane.” And in 1.24, they removed references to “master,” even from running clusters. This is the cause of our outage. Kubernetes node labels. Wow, so the word police brought down Reddit. Why on earth did someone think it a good idea to screw with existing names in running clusters in a cluster management tool?

The word police seem to have gone kinda quiet lately. I wonder if they realised that banning a few words wasn't really helping their cause.

Their main home seemed to be twitter.

I wonder if perhaps twitter's new ownership (and decrease in moderation) has impacted their activities? I wonder if perhaps it was an effort led by twitter employees because that sort of thing leads to greater use of twitter?

Re: Reddit Releases Post Mortem for Its 3 Hour Outage Last Week

#20

Post by wildzzz »

post_break wrote: post #19

Imagine if Cisco or Juniper decided to swap master or remove slave from their code. Core routers going down because of word police terminology and an admin who missed it in the change log.

Sounds more like admins need to read over change logs better and properly test updates on dev environments before just blindly updating systems. Features get deprecated, APIs update, crypto algorithms get dropped, it's entirely on the admin to ensure an update will actually work with existing code and systems.

On a very critical system, I wanted to use a newer python module that fixed a very annoying bug in the much older version we were running. Of course the module required a much newer version of python too. I upgraded everything and found that a function in a built-in module I had been using was entirely deprecated, very bad since it was used all over my code. I ended up writing my own module to overload the deprecated function into the new proper way of doing what I needed with only a simple change to my import statements. If I had just properly read over change logs and ran the update on a dev system, I wouldn't have any downtime since I could have made a fix early.

Post reply on HN