Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"DigitalOcean Killed Our Company"

twitter.com

Post reply on HN
Print view View original

321–330 of 620 posts

Re: "DigitalOcean Killed Our Company"

#321

Post by duskwuff »

andrewstuart wrote: post #246

Why is killing accounts part of the way they do business in any way, EVER? That's what destroys company reputation. I may be wrong but my understanding is that the gold standard - Amazon Web Services - will only ever suspend your account until an issue is sorted out. Whoever runs Digital Ocean needs to stand up and say very loud and clear to this community that they will never, ever delete accounts - if he doesn't th…

> Why is killing accounts part of the way they do business in any way, EVER?

Because fraud and abuse exist.

Sometimes customers really are doing malicious things which need to be stopped immediately, and the only thing that will make that happen is disabling their account. Trying to make accommodations for those users is a fast path to getting sued, getting blacklisted by mail providers, and/or losing your upstream connection entirely.

Re: "DigitalOcean Killed Our Company"

#322

Post by taffer »

harryh wrote: post #201
taffer wrote: post #169

Earlier quoted context omitted.

Caps or quotas are a better way to achieve this IMO. Random shutdowns do not make it sound as if DO is ready for production.

It's not just the volume of usage that can indicates fraud, but the pattern. In addition, relying on quotas creates a system that is easily games by perpetrators of fraud. Caps or quotas are not sufficient to deal with this problem.

If a cloud service is supposed to be ready for production, then customers should be safe to assume that they will not simply be shut down, especially not without warning. Otherwise, the provider must make clear that the service is only for hobby use and not for commercial use.

Re: "DigitalOcean Killed Our Company"

#323

Post by jdcro »

bcooks wrote: post #270

As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…

"In this particular scenario, we were slow to respond and had missteps in handling the false positive. This led the user to be locked out for an extended period of time."

This didn't seem like a case of being "too slow" - the customer in question went through your review process (which was slow, yes), and the only response he got was "We have decided not to reactivate your account, have a nice day".

That just seems like a lack of interest in supporting your customers that are falsely flagged.

Re: "DigitalOcean Killed Our Company"

#324

Post by elcritch »

> After sending multiple emails and DM on Twitter they unlocked our account, we got 12h of downtime and got a nice

Wait their entire business was effectively shutdown and all they did was send an email? Granted the DO handling of a possible abuse situation is shocking, but to allow your business to go down for 12 hours and not be trying to call any and every actual human being at DO seems to be negligent on their part. While us technical types love interfacing via digital means, some situations benefit from actually talking to a live human.

Re: "DigitalOcean Killed Our Company"

#325

Post by p2t2p »

duskwuff wrote: post #318
dahfizz wrote: post #302

Earlier quoted context omitted.

Again, I must disagree. If DO genuinely believed that you were doing something malicious and that data was harmful or evil for you to own (e.g. other people's SSN, etc) then they are in the "right" to deny access to it. DO should not be forced to aid bad actors. And, regardless of what DO should or should not do, they can do whatever they want with their own hard drives. You should structure your business accordingly…

For some practical, if extreme, examples: if a customer were to host a phishing site, or a site hosting CP, it would be grossly irresponsible (and likely even illegal) for the hosting provider to retain the customer's data after account suspension and allow them to download it.

When this happens they should contact law enforcement, not play god.

Re: "DigitalOcean Killed Our Company"

#326

Post by lelanthran »

walrus01 wrote: post #173
thaumaturgy wrote: post #166

Earlier quoted context omitted.

This race to the bottom has reached a point that it's harming customers. It's okay to be more expensive than the competition if you provide a better service.

Personal opinion, it's really important in the ISP/hosting world to identify what market categories are a race to the bottom, and if at all possible, refuse to participate in them. I look at companies selling $5 to $15/month VPS services and try to figure out how many customers they need to be set up for monthly recurring services, in order to pay for reasonably reliable and redundant infrastructure, and the math jus…

"you're responsible for doing all your own offsite backups"

That's going to be true no matter which cloud provider you choose.

Their ToS almost certainly include terms which allow them to kick you off and refund any monies for any reason whatsoever.

Good luck if you bought into their entire ecosystem and can't move elsewhere on a whim.

Re: "DigitalOcean Killed Our Company"

#327

Post by lapnitnelav »

bcooks wrote: post #270

As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…

Public post mortem? Brilliant.

Hope you can share what you learnt from this incident and hopefully you'll take a hard look at your processes.

I'd hate to be caught in the same issue, especially that we are already customers, and I'm not sure I'll have as much clout as Nicolas here to get your attention.

Re: "DigitalOcean Killed Our Company"

#328

Post by 6gvONxR4sf7o »

harryh wrote: post #201
taffer wrote: post #169

Earlier quoted context omitted.

Caps or quotas are a better way to achieve this IMO. Random shutdowns do not make it sound as if DO is ready for production.

It's not just the volume of usage that can indicates fraud, but the pattern. In addition, relying on quotas creates a system that is easily games by perpetrators of fraud. Caps or quotas are not sufficient to deal with this problem.

Caps and quotas limit the size of the chargeback.

Re: "DigitalOcean Killed Our Company"

#329

Post by perrygeo »

> Every 2-3 months we had to execute a python script that takes 1s on all our data (500k rows), to make it faster we execute it in parallel on multiple droplets ~10 that we set up only for this pipeline and shut down once it’s done.

Wait, a whole distributed computing sub-system to make a 1s process faster?

> I got their final message right after arriving in Portugal.

Did he initiate the script from a IP address outside France?

Though that might explain why DO's fraud detection was triggered, it doesn't excuse their actions. Send an email first, jeez.

Re: "DigitalOcean Killed Our Company"

#330

Post by nickpsecurity »

thaumaturgy wrote: post #114

Some people on HN hate Linode because of their past security screwups (which is valid), but having used both DO and Linode quite a lot, the support on Linode is way, way, way better than DO's. DO's tier 1 support is almost useless. I set up a new account with them recently for a droplet that needed to be well separated from the rest of my infrastructure, and ran into a confusing error message that was preventing it f…

I'll throw in Prgmr.com. One of their owners, Alyn Post, is on Lobsters with us. They even donate hosting to the site. He's been a super-nice guy over the years. Given how cost-competitive market is, they mainly differentiate on straight-forward offerings with good service. So, I tell folks about them if concerned about good service or more ethical providers.

https://prgmr.com/xen/

Main, potential caveat is they're Xen-based hosting. That may or may not matter depending on what one wants to run. They support the major Linux's.

Post reply on HN