Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"DigitalOcean Killed Our Company"

twitter.com

Post reply on HN
Print view View original

71–80 of 620 posts

Re: "DigitalOcean Killed Our Company"

#71

Post by fabian2k »

Hamuko wrote: post #19
ziddoap wrote: post #7

An expensive lesson learned in redundancy. It's a shame to see things play out this way, but sometimes a lesson is taught in a brutal fashion. I imagine the author will be more careful in the future regarding off-site backups, additional technical partners, contingency plans, etc.

There's a pretty hard cap on the level of redundancy you can do with a two-man company, as I assume a two-man company does not bring in a lot of money.

There are certainly a lot of limits on what you can do with such limited resources, but a reasonable backup with a different provider is certainly doable at that small scale.

It won't be entirely up to date when the worst case happens, you'll be unavailable and you'll probably have lost a day of data or so, but you won't have lost everything.

Rclone to AWS or Google or whatever is easy to set up, add a daily dump of your database to the folder you back up. Unless you handle a lot of data, costs are probably not a big factor.

Re: "DigitalOcean Killed Our Company"

#72

Post by freehunter »

el_programmador wrote: post #57
atonse wrote: post #6

I’m certainly getting downvoted for this. Not at all trying to blame the original posters and victims: While they seem great for hobbyist and small business sites, there’s no way I’d trust Fortune 500 client business to something like DigitalOcean. I just don’t see the benefits over a more established operation like AWS, Azure or GCP. Saving $50 here and there isn’t worth it.

That was true a long time ago, but recently DO has also proven to be a professional cloud hosting provider in the same league. Frankly, I was expecting the same level of support/service as AWS from them. BTW, I don't even see a downvote button around posts, there is only upvote button! Is it because I'm new here?

You need a certain number of points before you can see the downvote or flag options.

Re: "DigitalOcean Killed Our Company"

#73

Post by busterarm »

Hamuko wrote: post #19
ziddoap wrote: post #7

An expensive lesson learned in redundancy. It's a shame to see things play out this way, but sometimes a lesson is taught in a brutal fashion. I imagine the author will be more careful in the future regarding off-site backups, additional technical partners, contingency plans, etc.

There's a pretty hard cap on the level of redundancy you can do with a two-man company, as I assume a two-man company does not bring in a lot of money.

Their customers are Fortune 500 companies.

Clearly they should be charging more.

Re: "DigitalOcean Killed Our Company"

#74

Post by wpietri »

scarejunba wrote: post #54
zbobet2012 wrote: post #32

Earlier quoted context omitted.

They are, just don't rely one _one_ service. You need to be cloud agnostic and this is why.

Strong anti-recommendation for this advice. The cost in dev time plus keeping things running is not worth it. This is a rare event. Just use a more well-established cloud service.

That's generally good advice, but I think it's important to keep backups and other disaster-recovery stuff somewhere else. Heck, just buy a NAS box and sync things nightly.

Re: "DigitalOcean Killed Our Company"

#76

Post by vbezhenar »

They are reckless to trust a single company with their backups worth of money. I don't trust Google and Amazon with backups of my wedding video nobody cares about, so I'm using my home NAS with raid as a third additional storage (along with Glacier and Coldline). And they just trust DigitalOcean with all their data, assets? Crazy. Don't put all eggs in one basket. Register domain from one company, host DNS in another company, host your servers in third company, keep your backups in multiple places owned by multiple companies, test your backups periodically, have migration plan if one company decides to ban you.

That's another reason that I'm very skeptical of Amazon and Google offerings of their proprietary API. Rent virtual servers, no problems, you can rent those servers from a thousands of hosters. But if you're using their proprietary APIs, you will have to rewrite your software to migrate and you will have very tight time frame.

Re: "DigitalOcean Killed Our Company"

#77

Post by megous »

Hamuko wrote: post #19
ziddoap wrote: post #7

An expensive lesson learned in redundancy. It's a shame to see things play out this way, but sometimes a lesson is taught in a brutal fashion. I imagine the author will be more careful in the future regarding off-site backups, additional technical partners, contingency plans, etc.

There's a pretty hard cap on the level of redundancy you can do with a two-man company, as I assume a two-man company does not bring in a lot of money.

Well if you don't need HA, having a data backup (at least the most important data) outside the service is usually pretty cheap/easy.

Even the database replication is mostly setup and forget these days. Very reliable.

And they already did backups. It's just they did them to the wrong (same service) host.

Re: "DigitalOcean Killed Our Company"

#78

Post by tedivm »

atonse wrote: post #6

I’m certainly getting downvoted for this. Not at all trying to blame the original posters and victims: While they seem great for hobbyist and small business sites, there’s no way I’d trust Fortune 500 client business to something like DigitalOcean. I just don’t see the benefits over a more established operation like AWS, Azure or GCP. Saving $50 here and there isn’t worth it.

To be blunt I've heard of GCP doing this to other people before. AWS and Azure though both understand that customer service is extremely important, and shutting down services destroys confidence. In the case of them suspecting malicious activity they'd have actual security people look at what's happening, and then maybe blackhole their traffic while they start calling people.

After even a single incident like this no sane company would relay on DigitalOcean. This is the kind of crap you expect from a shared web host overselling resources, not from a company that wants to provide infrastructure to tech companies.

Re: "DigitalOcean Killed Our Company"

#79

Post by StudentStuff »

busterarm wrote: post #68
StudentStuff wrote: post #60

Earlier quoted context omitted.

Hmm, I've done lots of stuff on OVH that other hosts consider abusive, with nary a word of complaint from OVH. OVH seems to have this whole automation thing down cold, monitoring boxes for dying components and letting me run hog wild on their VMs without limit... The one time I used their phone support, the guy I got was fairly helpful. They seem like a hands off company overall.

If you get DDOS'd they will stick your VMs in a black hole until the traffic stops to protect their network, no matter how much you plead with them. Good practice, actually, but it's something people usually get frustrated with and they're literally famous for doing it.

This doesn't seem unreasonable, OVH is not advertising DDOS protection (nor do they seem to be structured to offer it). Some hosts will can your VM and throw out its data, which is a much worse outcome.

Edit: I stand corrected ¯\_(ツ)_/¯

Re: "DigitalOcean Killed Our Company"

#80

Post by GordonS »

ziddoap wrote: post #30
Hamuko wrote: post #24

Earlier quoted context omitted.

>It is clear they never considered a scenario where there DO account would be closed/compromised/down. I don't think the chance of DigitalOcean automatically freezing your account to a point where only a co-founder can do something about it has been well publicised.

In all practicality, DO freezing your account has the same effect of DO being down (or closing, etc.), or your account being compromised and you being locked out of it. A contingency plan should ideally have been in place for a scenario where, regardless of root cause, you have lost access to your DO account.

> In all practicality, DO freezing your account has the same effect of DO being down (or closing, etc.)

Given their size, that is extremely unlikely to happen without warning.

Post reply on HN