The great thing about DropBox is that I didn't even notice, despite using my files across the outage.
Outage post-mortem
31–40 of 44 posts
Re: Outage post-mortem
#32Couldn't upload files for more than a day. It was stuck at "connecting". I wish they had notified on twitter/blog upfront that they are still working on it.
Re: Outage post-mortem
#33I found it hard to work out where to get the most up to date information on the outage. I checked the blog, but the last message was their New Year message. In the app and on the main website (mobile version) I couldn't see anything... Glad I read HN otherwise I don't know how I would have come across this information. =)
Would be nice to have a service who stepped in when shit like this happened. I'd pay good money to have a tiger team appear out of thin air when the shit hit the fan.
I just meant to say that, it would be great if their mobile device application had some way of notifying users of an issue. Rather than just have a generic error when you try to upload files... it would have been good if the app had a banner that said they were experiencing issues and normal service would resume in x days... something.
Re: Outage post-mortem
#34This is, sadly, not a great post-mortem. They missed an opportunity for goodwill. I don't feel more confident in their level of understanding or ability to remediate the problems that led to it after having read it. I know they have an excellent engineering and operations staff -- this post-mortem doesn't reinforce that, though. A few of the things that jumped out at me after one reading: 1. The apology is the next t…
is this satire?
Re: Outage post-mortem
#35Couldn't upload files for more than a day. It was stuck at "connecting". I wish they had notified on twitter/blog upfront that they are still working on it.
About ten years ago, I had a minor epiphany while working on a system that used a standard client/service architecture. We were having "stability issues" with the back-end (trans: it was going down more often than a two dollar hooker). If the back-end service didn't responding, the GUI just hung; there was no indication for the end user that there was a problem - it just hung until it timed out (and because of the nature of the system, the timeout was relatively lengthy).
This caused a certain amount of consternation amongst our users (because they were losing money as a result), who would far prefer to be told that the system wasn't working than to have to wait for a timeout. So I got the GUI developers to add in a little bit of logic along the lines of:
if the response received from the back-end service starts with the string "ERROR_MESSAGE"
then parse and display the ensuing text to the user
else continue as normal
When the outage was over and we'd confirmed everything was working properly again, we'd would simply re-point the load-balancers back to the actual service IPs/ports, grab the file with all the usernames and email/call them to let them know that the service was back up. (We had planned to build more logic into the GUI and add an "Alert me when the system is back up" button to the error dialog, which would cause the GUI to automatically/silently re-try every X minutes and alert the user with a pop-up when full seervice was restored but we sorted out the stability problems before we got a chance to implement that).
Maybe Dropbox should implement something similar so that, instead of being stuck at "connecting", users get an error message.
Re: Outage post-mortem
#36This is, sadly, not a great post-mortem. They missed an opportunity for goodwill. I don't feel more confident in their level of understanding or ability to remediate the problems that led to it after having read it. I know they have an excellent engineering and operations staff -- this post-mortem doesn't reinforce that, though. A few of the things that jumped out at me after one reading: 1. The apology is the next t…
This post was just an incident review for a technology audience. Dropbox posted a separate apology to their users on their main blog: https://blog.dropbox.com/2014/01/back-up-and-running/ . The tone and detail seem totally appropriate since it ran concurrently with the other post.
Ah, that's interesting. I wasn't aware of this post at all, thanks for pointing it out.
Re: Outage post-mortem
#37Worked throughout the weekend, avid Dropbox user, didn't know they had a problem up until now.
Re: Outage post-mortem
#38The great thing about DropBox is that I didn't even notice, despite using my files across the outage.
Is that really a good thing?
What if you had finished up a document at home, thought it synced but it never did, and then showed up at a clients a few hours later wondering why you still had the old version on your phone/laptop?
Wouldn't you have wanted DropBox to let you know something was going on?
Re: Outage post-mortem
#39The great thing about DropBox is that I didn't even notice, despite using my files across the outage.
Is that really a good thing? What if you had finished up a document at home, thought it synced but it never did, and then showed up at a clients a few hours later wondering why you still had the old version on your phone/laptop? Wouldn't you have wanted DropBox to let you know something was going on?
> Wouldn't you have wanted DropBox to let you know something was going on?
This is such a hard thing to balance. I hate nagging notifications. Dropbox makes it really easy to see what the service status is. I glance at my menu bar/system tray icon, and look for the green check. If I don't see the green check, I know my docs haven't synced.
Re: Outage post-mortem
#40Earlier quoted context omitted.
Is that really a good thing? What if you had finished up a document at home, thought it synced but it never did, and then showed up at a clients a few hours later wondering why you still had the old version on your phone/laptop? Wouldn't you have wanted DropBox to let you know something was going on?
> Wouldn't you have wanted DropBox to let you know something was going on? This is such a hard thing to balance. I hate nagging notifications. Dropbox makes it really easy to see what the service status is. I glance at my menu bar/system tray icon, and look for the green check. If I don't see the green check, I know my docs haven't synced.
This is a glaring case of rose tinted glasses! Of course you'd want to know you were working on out of date src/document/etc...