Live data from Hacker News

Reddit crashed because of the growing subreddit blackout

theverge.com

11–20 of 148 posts

Re: Reddit crashed because of the growing subreddit blackout

#11

How long before they admins swoop in and change ownership of the blacked out subreddits?

What does that really accomplish? Reddit mods are often bad enough as is, imagine what Reddit scab mods would be like.

It would also be a huge invitation for the displaced the rest to make life unbearable for those new mods and the handful of admins overseeing them. Reddit as a community would make for one hell of an intellectual DDOS.

Re: Reddit crashed because of the growing subreddit blackout

#12

How long before they admins swoop in and change ownership of the blacked out subreddits?

I personally believe they will wait to see which ones remain shuttered. 48 hours is not very long to wait to maintain your free civilian labor.

After that, they'll definitely reclaim the shuttered subreddits with new moderators.

Re: Reddit crashed because of the growing subreddit blackout

#13

I have not read a reddit devblog or anything like that. My guess is they are doing some kind of caching of posts via elasticsearch/redis or something similar. It must depend on the privacy settings of a sub. So when 1000 subs go private, they potentially had to update hundreds of millions of comments. Anyone else have a theory?

the cynic in me says, they're being DDOSd (I mean, they pissed off millions of internet nerds who have their own definition of justice), and instead of admitting it they're using the opportunity to blame the subreddit mods.

then again, not sure why that messaging would be any better for them than admitting to a DDOS...

Re: Reddit crashed because of the growing subreddit blackout

#14

I have not read a reddit devblog or anything like that. My guess is they are doing some kind of caching of posts via elasticsearch/redis or something similar. It must depend on the privacy settings of a sub. So when 1000 subs go private, they potentially had to update hundreds of millions of comments. Anyone else have a theory?

That makes sense to me. It would be one variant of a larger theory of "a subreddit going private is an expensive process that is poorly optimized because it's normally so rare". Another variant would be that clicking the "go private" button might do something like start a huge cascade of queries/calls to mutate the status of individual posts and comments (and possibly hence also trigger the massive cache invalidation you mentioned). Possibly also "going private" infra, whatever it is, is some bespoke snowflake that doesn't autoscale (since it's never needed to and there was always something better to be doing with engineering time, or people just forgot).

One theory I have seen floated but which seems unlikely to me is that this was some kind of internal sabotage. I could maybe buy this if the protest was about a war or human rights issue or something, but I really don't think any IT pros would be willing to risk jail time for this one.

Re: Reddit crashed because of the growing subreddit blackout

#17
Could be due to home feeds not having enough cached content as a lot of the first page posts went private and had to be skipped? Would have led to have to go deeper in lists of posts to fill the first page and causing more cache misses and db reads and performance issues.

Re: Reddit crashed because of the growing subreddit blackout

#18

Earlier quoted context omitted.

Pretty sure Reddit uses Fastly as a CDN.

[flagged]

Can you elaborate? For us cloudflare has been incredibly reliable and incredibly cheap, although we'd consider moving if there was a better option.

I'm not aware of this perception, have you had issues with cloudflare? Does fastly have a better offering?

Re: Reddit crashed because of the growing subreddit blackout

#19

Earlier quoted context omitted.

[flagged]

Can you elaborate? For us cloudflare has been incredibly reliable and incredibly cheap, although we'd consider moving if there was a better option. I'm not aware of this perception, have you had issues with cloudflare? Does fastly have a better offering?

It was a joke.
Post reply on HN