I recently set up a website, and did consider AWS to host it. I decided not to, based on the problems Reddit had been having with AWS. I eventually got a VPS; looks like I made the right decision.
Reddit has now been down for 30+ hours
21–30 of 49 posts
Re: Reddit has now been down for 30+ hours
#22They are sort of limping back up now. Some users -- seems to be mostly gold users right now -- can get in.
It is reddit. Not experts-exchange. If it were "gold users can now login" that would be bad.
If they have to let users back in slowly (which they do), why shouldn't it be ones that contributed financially to the site?
Re: Reddit has now been down for 30+ hours
#23Being a sys admin and having to plan disaster recovery I can understand this scenario. I've played it out in my mind, its been done to death on paper and we've tested it. My question is, why aren't reddit doing anything about it? After numerous outages over the past few months it seems reddit's answer to everything is, "It's not our fault, it's Amazon". In all honesty without turning this into an attack that is ridic…
IIRC from previous posts on HN, you should correct that to read "Engineer of Reddit".
They're an organization in flux, and when you're in that state, the priority is survival.
Re: Reddit has now been down for 30+ hours
#24Earlier quoted context omitted.
Let me ask you. How does a company get into that state? Say you had 5 engineers, 1 billion page views a month and ongoing issues. Wouldn't you dedicate some resources to that? When one engineer leaves, wouldn't you worry a little? And then another, and another? Would you sit and do nothing? These events did not occur in a day. They occurred over a lengthy period of time. I can sympathize with the lone engineer becaus…
> JonnieCache mentioned Conde Naste will not let them hire more people. Yeah, I imagine the situation is more complicated than that, but look into some of the reddit blogposts and some of the admins comments on here for more details. There was a lot of stuff in the big 'AWS is down' thread about it, something about CN giving them an unlimited operating budget but a tiny, non negotiable hiring budget. God knows.
Gist I got from the 'AWS is down' thread was they had maxed their operation budget and that was why they couldn't afford to use multiple regions. Reddit being very write heavy the data synchronization across regions would be too expensive in both bandwidth and manpower.
Re: Reddit has now been down for 30+ hours
#25Being a sys admin and having to plan disaster recovery I can understand this scenario. I've played it out in my mind, its been done to death on paper and we've tested it. My question is, why aren't reddit doing anything about it? After numerous outages over the past few months it seems reddit's answer to everything is, "It's not our fault, it's Amazon". In all honesty without turning this into an attack that is ridic…
Reddit has 1 billion page views monthly let's cut them some slack.
Last I heard they had one full-time developer working on it, and if that is the case, I wouldn't be surprised if yesterday was their first day off in a awhile. Also, Quora was having the same difficulties and it mostly points to not doing what Twilio and a host of other architecture-aware teams did. Single points of failure will fail.
Re: Reddit has now been down for 30+ hours
#26Earlier quoted context omitted.
Reddit has 1 billion page views monthly let's cut them some slack.
Let me ask you. How does a company get into that state? Say you had 5 engineers, 1 billion page views a month and ongoing issues. Wouldn't you dedicate some resources to that? When one engineer leaves, wouldn't you worry a little? And then another, and another? Would you sit and do nothing? These events did not occur in a day. They occurred over a lengthy period of time. I can sympathize with the lone engineer becaus…
Conde Naste will not let them hire more people. Obviously we know where the fault lies then.
We don't know what's going on so I think it's a bit unfair to speculate.
But is there any evidence that Reddit is profitable? Would having a more stable platform honestly have much of an impact on that? Server issues aside, they seem to be doing pretty well in terms of page views and user acquisition.
Re: Reddit has now been down for 30+ hours
#27I recently set up a website, and did consider AWS to host it. I decided not to, based on the problems Reddit had been having with AWS. I eventually got a VPS; looks like I made the right decision.
Counterpoint: We've been happily using AWS for several years now. We haven't had one second of downtime. We host across multiple availability zones, and have been pretty insulated. We are hosted in the eastern region (which was the one having issues yesterday), but managed to get through everything ok. We did lose some instances in one AZ, but failed back to our other instances without any incident.
ANY provider is going to have issues, the AWS actually gives you a bunch of nice options to easily handle those. I'm not saying that everyone should use Amazon, but don't take Reddits issues (I'm not familiar enough with their architecture to know where the fault really lies) to be the only data point for AWS reliability.
Re: Reddit has now been down for 30+ hours
#28Earlier quoted context omitted.
It is reddit. Not experts-exchange. If it were "gold users can now login" that would be bad.
If they have to let users back in slowly (which they do), why shouldn't it be ones that contributed financially to the site?
If it has to be based on a metric, it should be based on the Karma. "Who contributed the most for the site."
Re: Reddit has now been down for 30+ hours
#29Is there any proof that Conde Nast is trying to starve-off Reddit so they can maybe write it off on their taxes? They just don't seem to have a single ounce of pride in owning it.
It's amazing how much has been done with it's development given how little they have to work with.
Re: Reddit has now been down for 30+ hours
#30Maybe it's just me. I feel HN has become very sluggish this morning. Does that mean lots of redditors come over here? Edit: Wierd, got downvoted. I do think this is related to the topic, so the failure of Amazon can have a domino effect, say, AWS fails -> Reddit fails -> HN sluggish/struggling -> ... Some times I really question the resilience of the social networks, as most of them may not be designed to handle 2x o…
> Does that mean lots of redditors come over here?
I know I did.