Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"DigitalOcean Killed Our Company"

twitter.com

Post reply on HN
Print view View original

121–130 of 620 posts

Re: "DigitalOcean Killed Our Company"

#121

Post by ar0 »

StudentStuff wrote: post #79
busterarm wrote: post #68

Earlier quoted context omitted.

If you get DDOS'd they will stick your VMs in a black hole until the traffic stops to protect their network, no matter how much you plead with them. Good practice, actually, but it's something people usually get frustrated with and they're literally famous for doing it.

This doesn't seem unreasonable, OVH is not advertising DDOS protection (nor do they seem to be structured to offer it). Some hosts will can your VM and throw out its data, which is a much worse outcome. Edit: I stand corrected ¯\_(ツ)_/¯

Where are you getting this from? OVH very prominently advertises their DDOS protection that comes with all their products (https://www.ovh.com/ca/en/en/anti-ddos/faq.xml). In fact, they once used it to explain why they increased the prices of their VPSes (https://community.centminmod.com/threads/ovh-increase-prices...).

And they also have a lot of experience with it because they host a lot of game servers which are very prone to DDoS attacks (see e.g. https://securityaffairs.co/wordpress/51640/cyber-crime/tbps-...), so they are definitely “structured to offer it”.

Re: "DigitalOcean Killed Our Company"

#122

Post by stingraycharles »

_bxg1 wrote: post #101
macspoofing wrote: post #89

Earlier quoted context omitted.

>That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places Yes they should. How many 2-man shops do you think follow all the proper backup and security procedures?

It could literally be a cron job that dumps your DB to a desktop computer once a week. Not exactly CIA-level stuff.

More realistically they would have done backups inside DO and would still be locked out. Not many people actually do complete offsite backups to a completely different hosting provider, getting locked out of your account is usually just not a consideration. It’s unrealistic to expect this of a tiny startup.

Re: "DigitalOcean Killed Our Company"

#123

Post by hyperpape »

jerf wrote: post #42
duxup wrote: post #2

It's interesting how many companies simply shut down service rather than say give a warning and wait for a response (or at least start a clock). Granted that would require people to communicate and use some form of reason. Even DMCA for all its warts fires up a warning and has a response mechanism (granted other issues there).

"It's interesting how many companies simply shut down service rather than say give a warning and wait for a response (or at least start a clock)." I'm sure many people have started their companies firmly convinced that they'll give plenty of warnings and never automatically shut anything down. The problem is, you rapidly discover that doesn't scale, not even on a human level. You send your notice. 48 hours later, you…

Your point is reasonable in many cases. But in this particular case, the charges would not have been substantially bigger than what this company paid before, and an automated system ought to take that into account.

If I’m a $10/month customer, kill my account early, and it’ll save me more often than not. If I’m a big spender, maybe wait a bit longer.

Re: "DigitalOcean Killed Our Company"

#124

Post by z3t4 »

_bxg1 wrote: post #64

Looks like Moisey Uretsky personally intervened fairly quickly: https://twitter.com/moiseyuretsky/status/1134547532149854208 That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places. If they'd had the data, they could have spun up their service on a different hosting provider relatively easily.

Having backups in two places could easily triple the hosting costs. The question is what costs more. Eg. Losing data vs backup costs.

Re: "DigitalOcean Killed Our Company"

#125

Post by millettjon »

thaumaturgy wrote: post #114

Some people on HN hate Linode because of their past security screwups (which is valid), but having used both DO and Linode quite a lot, the support on Linode is way, way, way better than DO's. DO's tier 1 support is almost useless. I set up a new account with them recently for a droplet that needed to be well separated from the rest of my infrastructure, and ran into a confusing error message that was preventing it f…

+1 I have used both as well. Linode support has always been good and DO support has been a blocking barrier.

Re: "DigitalOcean Killed Our Company"

#126

Post by CharlesColeman »

badrequest wrote: post #109
macspoofing wrote: post #89

Earlier quoted context omitted.

>That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places Yes they should. How many 2-man shops do you think follow all the proper backup and security procedures?

"2-man teams generally don't prioritize backups" isn't an excuse for not prioritizing backups.

> "2-man teams generally don't prioritize backups" isn't an excuse for not prioritizing backups.

They had backups, but being arbitrarily cut-off from their hosting provider wasn't part of their threat model.

Isn't a big part of cloud marketing the idea that they're so good at redundancy, etc. that you don't need to attempt that stuff on your own? The idea that you have to spread your infrastructure across multiple cloud hosting providers, while smart, removes a lot of the appeal of using them at all. In any case, it's also probably too much infrastructure cost for a 2-man company.

Re: "DigitalOcean Killed Our Company"

#127

Post by thaumaturgy »

_bxg1 wrote: post #101
macspoofing wrote: post #89

Earlier quoted context omitted.

>That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places Yes they should. How many 2-man shops do you think follow all the proper backup and security procedures?

It could literally be a cron job that dumps your DB to a desktop computer once a week. Not exactly CIA-level stuff.

That's better than nothing, but still not great.

We don't know the structure of their DB and whether failover is important or not, so we don't know if the DB can be reliably pulled as a flat file backup and still have consistent data.

We also don't know how big the dataset is or how often it changes. Sometimes "backup over your home cable connection" just isn't practical.

Cron jobs can (and do) silently fail in all kinds of annoying and idiotic ways.

And as most of us are all too painfully aware, sometimes you make less-than-ideal decisions when faced with a long pipeline of customer bug reports and feature requests, vs. addressing the potential situation that could sink you but has like a 1 in 10,000 chance of happening any given day.

But yes, granted that as a quick stop-gap solution it's better than nothing.

Re: "DigitalOcean Killed Our Company"

#128

Post by cpeterso »

wpietri wrote: post #82

Does anybody know what they actually do? I dug around to find a Wayback version that actually had text. I understand the words on their own, but when put together I get nothing: https://web.archive.org/web/20181030015237/https://raisup.co...

Automated off-site backups for startups

Re: "DigitalOcean Killed Our Company"

#129

Post by MrStonedOne »

I see no need for DO to shut down accounts and active long established nodes for a fraud check. Disable the offending node, disable making new ones, and limit editing existing ones to off/on controls.

Regardless of this example of a false positive, locking whole accounts over that is unwise.

Re: "DigitalOcean Killed Our Company"

#130

Post by coldtea »

>We lost everything, our servers, and more importantly 1 year of database backups. We now have to explain to our clients, Fortune 500 companies why we can’t restore their account.

I think what you have to explain is why there wasn't a contingency plan, with your own servers, colocation, another cloud offering, etc...

Post reply on HN