I wonder what Reddit devops is like, I find it so strange that such a trafficked website can’t handle rollouts without what appears (on the outside) to be a full reboot. And also strange that the rollout takes so long. Makes me think they are doing something naive like running on naked vms and having one guy running a script to take them down, update os, redeploy new binary, etc. all at once. Then probably encountering issues and live debugging them until it works again
Reddit was down for over two hours
81–90 of 283 posts
Re: Reddit was down for over two hours
#82Earlier quoted context omitted.
Every user engagement metric has increased since the redesign. Unfortunate for some of us, and it doesn't preclude the metrics measuring the wrong thing, but that's reality on the ground. Same reason why Gmail is a bloated mess, because that's what the metrics show is wanted by users.
> Every user engagement metric has increased since the redesign. Doesn't that just mean that users are spending more time on the site because it's so much more of a pain to get what they actually want out of it? Like, you could increase "user engagement" of a shopping mall by turning it into a labyrinth and rearranging the shops every day.
Reddit is a corporation with a large amount of VC investment. Profit is a primary goal, and engagement that shows more time and thus more revenue is exactly what they want.
Re: Reddit was down for over two hours
#83Earlier quoted context omitted.
Except UX is everything. 100s of billions in corporate value are based on nothing more than better UX. If only everyone who talks about usenet, IRC, and the rest could actually be bothered to update the experience to what modern users demand.
You know, the people who talk about USENET and IRC are people who don't want to be "updated" to what modern users demand. I don't know if you've had a look around you lately, but the modern internet is a dumpster fire. Anyways, ideally you would be able to connect to services with whichever client you want. You could use a pretty eye candy client, or then not. With the web, however, it's an impossible feat to impleme…
> the people who talk about USENET and IRC are people who don't want to be "updated" to what modern users demand.
IRC is objectively terrible. If you ask a question, the only way to get the answer is if you stay logged in until someone answers you. That's like buying a phone that doesn't ring unless you're already holding it up to your ear, or an email client that deletes all your email unless you happen to be looking at your inbox at the moment a new message arrives.
Re: Reddit was down for over two hours
#84Reddit is the only major social media/aggregator I can think of that actually has to go offline for scheduled maintenance still. I can’t imagine the kind of shitshow it must be internally. Imagine if Facebook or Twitter had to go down regularly for “maintenance”. Heads would roll.
Steam is worse - weekly "maintenance" on Tuesday evening, where everyone is disconnected for ~10-20 minutes. It's common knowledge not to play at that time if the game uses Steam networking, otherwise you'll be kicked mid-session with a "Lost connection to Steam" error message and lose all your progress. And yes, this is happening in 2019.
At least it’s scheduled so you can schedule around it
Re: Reddit was down for over two hours
#85Earlier quoted context omitted.
Don't forget to turn off call waiting. I always forgot to turn off call waiting....
I'm showing my youth here, but this is a legitimate question. Isn't call waiting just a thing that exists on the line? Did phones used to have a switch to toggle that? And what was the harm in keeping it on? When I had dial up, I don't recall having to consider call waiting.
The incoming call tone would disrupt modem traffic so you had to dial a code prior to the number you wanted to dial to disable this temporarily for the duration of your outgoing call.
Re: Reddit was down for over two hours
#86Earlier quoted context omitted.
Every user engagement metric has increased since the redesign. Unfortunate for some of us, and it doesn't preclude the metrics measuring the wrong thing, but that's reality on the ground. Same reason why Gmail is a bloated mess, because that's what the metrics show is wanted by users.
> Every user engagement metric has increased since the redesign. Doesn't that just mean that users are spending more time on the site because it's so much more of a pain to get what they actually want out of it? Like, you could increase "user engagement" of a shopping mall by turning it into a labyrinth and rearranging the shops every day.
Not necessarily for social media where there’s an endless river of entertainment content. I doubt most people go to Reddit for any specific purpose or task they’re trying to accomplish as quickly as possible.
Re: Reddit was down for over two hours
#87Earlier quoted context omitted.
Don't forget to turn off call waiting. I always forgot to turn off call waiting....
I'm showing my youth here, but this is a legitimate question. Isn't call waiting just a thing that exists on the line? Did phones used to have a switch to toggle that? And what was the harm in keeping it on? When I had dial up, I don't recall having to consider call waiting.
Dialing *67, I think, in the dial string for dialup turned off call waiting for the duration of the call. Otherwise a second call would try to ring through and disrupt your Super Z-modem download.
Re: Reddit was down for over two hours
#88Earlier quoted context omitted.
Don't forget to turn off call waiting. I always forgot to turn off call waiting....
I'm showing my youth here, but this is a legitimate question. Isn't call waiting just a thing that exists on the line? Did phones used to have a switch to toggle that? And what was the harm in keeping it on? When I had dial up, I don't recall having to consider call waiting.
[deleted]
Re: Reddit was down for over two hours
#89Earlier quoted context omitted.
Every user engagement metric has increased since the redesign. Unfortunate for some of us, and it doesn't preclude the metrics measuring the wrong thing, but that's reality on the ground. Same reason why Gmail is a bloated mess, because that's what the metrics show is wanted by users.
I don't disagree, but "engagement" does not necessarily reflect value to the user. Taking more time to accomplish one's goal (e.g., obtain information) than necessary may improve engagement metrics from the service's perspective, but from the user's perspective, that's extra time that might have been wasted and better spent on other things.
Maybe not value to each individual user, but in aggregate the numbers of users and content are increasing. Collective value and network effects are higher. This is also leading to more revenue which is critical for a corporation.
I'm sure they lose some users but they've gained far more than they've lost so far.
Re: Reddit was down for over two hours
#90Earlier quoted context omitted.
So what? 24/7 builds up to fragility like the NYC subway system or has huge costs. It's social media, not a hospital.
It's more the fact that the engineers working there should be embarrassed, not that it's important. You would think that the engineers there would be able to improve their infrastructure after a few years but it's still as flaky now as it was a few years ago. That doesn't speak well of the caliber of engineers there.
That's unfair. Engineers don't always get to exercise their skills or best practices if their CTO gets in the way. Or if other external obstacles block the CTO. I've worked with very talented people who are totally wasted because the company just doesn't care to get the tech right when it comes down to making that investment.
Autonomy to get things right when it's the right call is sort of a rarity and a real luxury in my experience. I've built a ton of things I knew could have been built better if the rest of the company saw the value in doing so.
When given the option to do your job as asked or quit because you don't have your way, I'd rather just do my job and wait for opportunities to improve things if they present themselves.