Reddit crashed because of the growing subreddit blackout • Hacker News Board

Re: Reddit crashed because of the growing subreddit blackout

#31

Post by falcolas » Mon, Jun 12, 2023, 8:32 PM UTC

phpisthebest wrote: post #28

falcolas wrote: post #12

Earlier quoted context omitted.

I personally believe they will wait to see which ones remain shuttered. 48 hours is not very long to wait to maintain your free civilian labor. After that, they'll definitely reclaim the shuttered subreddits with new moderators.

Reddit is odd.... while most communities hate the mods, they hate admins far more. Admins taken over a subreddit would kill it in the eyes of the users.

I think the admins will care more about the appearance of "business as usual" in the face of an IPO than the opinions of the users who would notice a moderator replacement.

In their eyes, this is all nothing more than a tempest in a teapot; a small pothole on the road to public investor dollars.

Re: Reddit crashed because of the growing subreddit blackout

#32

Post by 2OEH8eoCRo0 » Mon, Jun 12, 2023, 8:32 PM UTC

ElevenLathe wrote: post #14

Capricorn2481 wrote: post #2

I have not read a reddit devblog or anything like that. My guess is they are doing some kind of caching of posts via elasticsearch/redis or something similar. It must depend on the privacy settings of a sub. So when 1000 subs go private, they potentially had to update hundreds of millions of comments. Anyone else have a theory?

That makes sense to me. It would be one variant of a larger theory of "a subreddit going private is an expensive process that is poorly optimized because it's normally so rare". Another variant would be that clicking the "go private" button might do something like start a huge cascade of queries/calls to mutate the status of individual posts and comments (and possibly hence also trigger the massive cache invalidation…

Would a better protest be rapidly switching subreddits from public/private?

Re: Reddit crashed because of the growing subreddit blackout

#33

Post by putnambr » Mon, Jun 12, 2023, 8:32 PM UTC

Capricorn2481 wrote: post #2

I have not read a reddit devblog or anything like that. My guess is they are doing some kind of caching of posts via elasticsearch/redis or something similar. It must depend on the privacy settings of a sub. So when 1000 subs go private, they potentially had to update hundreds of millions of comments. Anyone else have a theory?

My theory is that some middleware applications have policies to scale to zero or one based on traffic patterns, when scaled down due to lack of traffic things start breaking.

Re: Reddit crashed because of the growing subreddit blackout

#34

Post by Scoundreller » Mon, Jun 12, 2023, 8:32 PM UTC

philip1209 wrote: post #5

Capricorn2481 wrote: post #2

I have not read a reddit devblog or anything like that. My guess is they are doing some kind of caching of posts via elasticsearch/redis or something similar. It must depend on the privacy settings of a sub. So when 1000 subs go private, they potentially had to update hundreds of millions of comments. Anyone else have a theory?

Cache invalidation makes sense. Could also be some kind of fanout query that assumes the top subreddits are open and wastes CPU cycles.

I’ve noticed that my comment history in privatized subreddits are now unavailable, so I wonder if it’s related to that.

Re: Reddit crashed because of the growing subreddit blackout

#35

Post by evan_ » Mon, Jun 12, 2023, 8:33 PM UTC

bitshiftfaced wrote: post #21

EA-3167 wrote: post #11

Earlier quoted context omitted.

What does that really accomplish? Reddit mods are often bad enough as is, imagine what Reddit scab mods would be like. It would also be a huge invitation for the displaced the rest to make life unbearable for those new mods and the handful of admins overseeing them. Reddit as a community would make for one hell of an intellectual DDOS.

I seem to remember someone saying something about how a small number of users mod the vast majority of Reddit in terms of the number of users within the subreddits they control. I wouldn't be surprised if there was a black market for mod status. Might not be such a bad thing if the establishment was totally uprooted.

> a small number of users mod the vast majority of Reddit

I think it's more like there are a small number of users who are mods on a huge number of subs, but there are other mods that focus more directly on the actual moderation of those subs.

Re: Reddit crashed because of the growing subreddit blackout

#36

Post by schaefer » Mon, Jun 12, 2023, 8:33 PM UTC

sofixa wrote: post #29

> A significant number of subreddits shifting to private caused some expected stability issues, and we’ve been working on resolving the anticipated issue,” Calling bullshit on that one. If they had expected stability issues, and knew exactly which subreddits will go private and when, how is it possible they had so much "anticipated issues"?

The world has finite resources... such as man-hours.

Re: Reddit crashed because of the growing subreddit blackout

#37

Post by subarctic » Mon, Jun 12, 2023, 8:34 PM UTC

It'd be funny if the engineers at reddit decided now was a good time to upgrade postgres or do some other kind of scheduled maintenance, and just not worry about downtime because the blackout is on anyway.

Re: Reddit crashed because of the growing subreddit blackout

#38

Post by falcolas » Mon, Jun 12, 2023, 8:35 PM UTC

sofixa wrote: post #29

> A significant number of subreddits shifting to private caused some expected stability issues, and we’ve been working on resolving the anticipated issue,” Calling bullshit on that one. If they had expected stability issues, and knew exactly which subreddits will go private and when, how is it possible they had so much "anticipated issues"?

I imagine a previously not-stress-tested component that has to do with searches/fetches with private subreddits went kaboom.

Not too much of a stretch, IMO.

A fantastic Schadenfreude generator, regardless.

Re: Reddit crashed because of the growing subreddit blackout

#39

Post by jedberg » Mon, Jun 12, 2023, 8:35 PM UTC

sofixa wrote: post #29

> A significant number of subreddits shifting to private caused some expected stability issues, and we’ve been working on resolving the anticipated issue,” Calling bullshit on that one. If they had expected stability issues, and knew exactly which subreddits will go private and when, how is it possible they had so much "anticipated issues"?

My guess is that they knew the caches would get blown out and would have to refill with content from non-blacked out subs, and decided to just take the hit and let it ride, knowing it would recover itself soon enough.

I mean, honestly that's probably what I would have done if I were still in charge.

Re: Reddit crashed because of the growing subreddit blackout

#40

Post by this_user » Mon, Jun 12, 2023, 8:35 PM UTC

stillbourne wrote: post #8

How long before they admins swoop in and change ownership of the blacked out subreddits?

A long time IMO, because that would just escalate the situation. At this point they are going to wait and see in the hopes that the majority of people lose interest after a few protests. Then they can just push through the changes.

If they take over these subs, you just radicalise the users, and you now have to find new moderators for these subs. In a worst case scenario, users actually start migrating to another platform.