Live data from Hacker News

"DigitalOcean Killed Our Company"

twitter.com

181–190 of 620 posts

Re: "DigitalOcean Killed Our Company"

#181
I think, going forward, a data backup is not going to be enough. You’ll need a full devops plan b/c where you just redeploy to another hosting provider.

Even better: run your infrastructure across multiple hosting providers with something like consul. DO might cause slower service but not a death sentence.

Re: "DigitalOcean Killed Our Company"

#182
post #101

Earlier quoted context omitted.

>That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places Yes they should. How many 2-man shops do you think follow all the proper backup and security procedures?

It could literally be a cron job that dumps your DB to a desktop computer once a week. Not exactly CIA-level stuff.

Sure it could be. Still not enough companies actually do this though...

Re: "DigitalOcean Killed Our Company"

#183

To everyone justifying the lack of a backup strategy by saying they’re a two man show: As far as sympathy goes, you’re not wrong. But you’re also justifying every pain in the ass procurement process you’ve ever dealt with. Your attitude is why so many companies won’t go near a two man shop.

I would agree if they were reckless, but here they had a backup strategy, just the risk of a complete sudden shutdown from their provider was not taken into it. This is a new risk that must be taken into account, and I’m not sure procurement would have scanned this. Also the pain in procurement is usually because it’s in fact on irrelevant arbitrary administrative things and not on real risks like this.

Re: "DigitalOcean Killed Our Company"

#184
post #64

Looks like Moisey Uretsky personally intervened fairly quickly: https://twitter.com/moiseyuretsky/status/1134547532149854208 That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places. If they'd had the data, they could have spun up their service on a different hosting provider relatively easily.

>That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places Yes they should. How many 2-man shops do you think follow all the proper backup and security procedures?

I feel for these guys, but that's not "all the proper backup procedures". I'm part of a three-man shop and storing backups in another place is the second thing you do immediately after having backups in the first place. Never mind being locked out by the company - what happens if the data centre burns to the ground?

Re: "DigitalOcean Killed Our Company"

#185
post #34
post #8

Earlier quoted context omitted.

This is obvious. Start your app on Digital Ocean or Heroku, and when it becomes a viable business get that shit on a grown up platform.

I can't speak on Digital Ocean and I guess it depends how you define "viable" but I've been running my upper six-figure revenue business on Heroku with tremendous success over the past 6 years. Customer support is super responsive and very helpful. The minimal downtime I've faced had has largely been the fault of AWS.

Couldn't agree more! We've been using Heroku at ReadMe for ~5 years and it is easily my favorite piece of technology we use. Off the top of my head I can't think of a single issue that entire time that has been their fault.

Re: "DigitalOcean Killed Our Company"

#186
post #143

Earlier quoted context omitted.

Linode is definitely in the minority here. Most companies, in tech and outside of it, seem to follow the DO model. Twitter provides decent service, and the official help channels provide canned responses and template emails. I somewhat blame people in tech, actually. More than one company is creating products that "cut customer service costs via machine learning", which is code for "pick keywords from incoming ticket…

There’s nothing wrong with that approach, the person raising the support ticket likely hasn’t read through all the documentation of the product they’re using.

If implemented well, sure -- sometimes, maybe often, you can point a customer to a support document that directly answers their specific question and relieves some of the load on your staff. That's great.

But the execution matters a lot, and DO's is currently not great. IIRC, it takes clicking through a few screens of "are you sure your question isn't in our generic documentation? How about this page? No? This one then? Still no? You're really sure you need to talk to someone about this error? sigh Okay, fine then."

These systems should not be implemented as a barrier to reaching human support, but they often are.

Re: "DigitalOcean Killed Our Company"

#187
This exact same thing happened to me last year. I accessed my account abroad and they perma banned me.

Support was useless and even with evidence did not believe who I was.

I then somehow convinced them to give me temp access, which in my opinion is even worse. They didn't believe me about who I was and then gave me temporary access to an account. DO can't be trusted when their support team could so easily be socially engineered.

Re: "DigitalOcean Killed Our Company"

#188

To everyone justifying the lack of a backup strategy by saying they’re a two man show: As far as sympathy goes, you’re not wrong. But you’re also justifying every pain in the ass procurement process you’ve ever dealt with. Your attitude is why so many companies won’t go near a two man shop.

Beautifully put.

Re: "DigitalOcean Killed Our Company"

#189

Some people on HN hate Linode because of their past security screwups (which is valid), but having used both DO and Linode quite a lot, the support on Linode is way, way, way better than DO's. DO's tier 1 support is almost useless. I set up a new account with them recently for a droplet that needed to be well separated from the rest of my infrastructure, and ran into a confusing error message that was preventing it f…

Long-time Linode customer here. What were the security screwups? I tried a couple searches but didn't turn up anything.

Re: "DigitalOcean Killed Our Company"

#190
I think a real takeaway here is to avoid going all in on one hosting provider. You should always have "off-site" backups for mission critical data.

Yes, it sucks that DO did this. But this is hardly the first time someone got screwed over by some poor AI automated security. Backing up your data to backblaze or AWS would be the cheapest insurance policy you could buy.

Post reply on HN