Live data from Hacker News

DigitalOcean lost our server

murze.be

211–219 of 219 posts

Re: DigitalOcean lost our server

#211
post #152

I'm more worried about the following post in that thread: -------------------------------------------------------- We were loyal customers of DigitalOcean for over 2 years. We showed up to work one morning and had a client email stating that their website was down. We checked, sure enough. We tried logging in to our DI account and it said it was suspended. After searching around, we realized our CC had expired. No bi…

Check out the followup:

Hey Matthew,

I'm Zach, Director of Support here at DigitalOcean. Thanks for raising this topic.

I was able to locate your account, and I see that I was the one who granted the credit and followed up via email. I hadn't heard back from you until now, and I'm happy that we have an open line of communication. I'm hopeful that this thread starts a conversation, that it clarifies what steps we take, and we might even uncover a different solution that works well for everyone.

To start with, I want to be really clear, this is the absolute worst case scenario. I absolutely don't want it to happen to any customer, let alone someone who has been with us for such a long time. There's really a delicate balance that we must strike between customers who forget to pay and customers who do not want to pay.

Currently, our notifications for overdue balances are sent via email. If a customer account is unpaid on the first of the month, we send ~15 emails total notifying the customer of the situation, and subsequently power off servers 21 days after the account is on hold as a further way to gain your attention. At this time, droplets are removed from your account 14 days following power off, which is an increased amount of time from what you experienced. It was increased from 3 days based on past user feedback.

Why do we do it this way? In the past, we had no hold or suspension process for non-payment, which enabled bad actors to run for months and months without paying. From a business perspective, we made a decision to put a scalable process in place that limited how long an account could go unpaid.

As a support team and business, we are always willing to work with our customers who are unable to pay. We are always available via ticket, our contact form, and have made wide-ranging attempts to help users who aren't able to pay due to banking regulations: https://insidedigitalocean.com...

I'd like some input, so I'll publicly ask for feedback on questions that I've asked privately before. I'd also like any other feedback that we can consider on how to make this a more positive experience for everyone

-Knowing that we do not do phone support, what's the best way to notify you of a past-due balance? -Is SMS effective at times like this?

I would love to hear your thoughts.

Thank you, Zach zach@digitalocean.com

Re: DigitalOcean lost our server

#212
post #174
post #167

Earlier quoted context omitted.

The world where employees care more about management satisfaction than customer satisfaction? And it's difficult to prove that bothering some customers here and there is impacting corporate revenue. Policies like this slip through the cracks.

Yes, it's digital oceans fault that people want to pay bargain basement prices and then wonder why they are getting bargain basement service.

Sorry, that doesn't excuse the purely bad idea of scripting deletion of data at the first sign of trouble.

Re: DigitalOcean lost our server

#213
post #174

Earlier quoted context omitted.

Yes, it's digital oceans fault that people want to pay bargain basement prices and then wonder why they are getting bargain basement service.

Sorry, that doesn't excuse the purely bad idea of scripting deletion of data at the first sign of trouble.

It's not at the first sign of trouble, though. It's after ~15 email notifications over the course of over a month, and no contact back.

Re: DigitalOcean lost our server

#214

Earlier quoted context omitted.

If you have used shared hosting you know as soon as you pay for your expired account, everything is back as it was. Never thought this was a problem until you try to test how long you can go until you pay and still restore you data. If DO's CC can't handle data management problems they probably will connect you to their engg/ops team. So I would assume they would prefer having it resolved ( by giving the data if they…

I agree with the first paragraph. The second paragraph hinges on the rationale that if customer data were available, then CS (Customer Service) agents would have made every effort to contact the ops/engineering team to retrieve those to avoid a social disaster. Unfortunately, this isn't true in the companies that I worked for, which are big customer-facing MNCs. First of all, for US consumers, the CS teams most likel…

For the record, that's not how it works at DO. The support department encompasses Platform Support (like T1), Trust & Safety (handles cases of possible fraud or abuse complaints), and CloudOps (sysadmins who help maintain the cloud).

I'm on Platform Support and while I can't access the actual hypervisor hardware like CloudOps can, we do not have "scripts", and we always do our best to help customers in any way we can. We also have a myriad of tools that we can use to monitor our platform and help troubleshoot any issues that may arise in our tickets.

My roommate and bestie is on CloudOps, and she sits right next to me at our HQ in NYC. While we have a great remote culture, all of us on support are in the US, with the exception of one guy in London (who started last week and is AWESOME).

Not only that, but I work very closely with all of the other departments here at DO, and our executives are extremely accessible as well. If we need to pass on feedback, we always do so.

We're also encouraged to go out of scope and do whatever possible to help out our customers, and I wouldn't change a thing. I love what I do, I love the people I help, and I love my coworkers.

Re: DigitalOcean lost our server

#215
post #152

I'm more worried about the following post in that thread: -------------------------------------------------------- We were loyal customers of DigitalOcean for over 2 years. We showed up to work one morning and had a client email stating that their website was down. We checked, sure enough. We tried logging in to our DI account and it said it was suspended. After searching around, we realized our CC had expired. No bi…

>We've been a customer for 2 years, surely they could pick up the phone and call us. I don't understand that reasoning from customers to be honest. That right there, throws away the $5 and $10 plans. Having some one trying to contact a customer, just because their credit card expired, cost more than the actual hosting. Top tier service does comes from companies with $5 a month plans. I'm sorry it just doesn't.

Because $5/$10/month is not the same thing as Customer Lifetime Value.

Re: DigitalOcean lost our server

#216

Earlier quoted context omitted.

Sorry, that doesn't excuse the purely bad idea of scripting deletion of data at the first sign of trouble.

It's not at the first sign of trouble, though. It's after ~15 email notifications over the course of over a month, and no contact back.

Both anecdotes mentioned timeframes of about 15 days after longer than a year of service.

Re: DigitalOcean lost our server

#217
post #152

I'm more worried about the following post in that thread: -------------------------------------------------------- We were loyal customers of DigitalOcean for over 2 years. We showed up to work one morning and had a client email stating that their website was down. We checked, sure enough. We tried logging in to our DI account and it said it was suspended. After searching around, we realized our CC had expired. No bi…

Check out the followup: Hey Matthew, I'm Zach, Director of Support here at DigitalOcean. Thanks for raising this topic. I was able to locate your account, and I see that I was the one who granted the credit and followed up via email. I hadn't heard back from you until now, and I'm happy that we have an open line of communication. I'm hopeful that this thread starts a conversation, that it clarifies what steps we take…

3 days seemed like an accident waiting to happen.

This updated policy seems a lot better; it would probably be great to ask people for secondary contact information such as SMS for emergencies.

Re: DigitalOcean lost our server

#218

Earlier quoted context omitted.

It's not at the first sign of trouble, though. It's after ~15 email notifications over the course of over a month, and no contact back.

Both anecdotes mentioned timeframes of about 15 days after longer than a year of service.

The timeframe is 3 weeks until suspension, then another 2 weeks until deletion. At the time this user was affected, it was 3 weeks until suspension, then 3 days until deletion, but that period was raised to 2 weeks due to feedback that 3 days wasn't enough.

I'm not saying that excuses things, but it's far from "at the first sign of trouble".

Re: DigitalOcean lost our server

#219
post #177

Earlier quoted context omitted.

I'm looking for a good host, was considering DO, but now I'm not. Any suggestions?

Needs vary, but here's a brief list of competitors: * OVH.com — huge, great price point and performance * Linode.com — also huge, favorite of many developers * Vultr.com — owned by Choopa (bigger player) * RamNode.com — doesn't scream "super modern", but reviews are great There are many others, but those are off the top of my head that correlate with DO.

- leaseweb

- 1andone

Vote against linode here, yes, they're large but they are also not all that good.

Post reply on HN