"DigitalOcean Killed Our Company" • Hacker News Board

Re: "DigitalOcean Killed Our Company"

#581

Post by sneak » Sat, Jun 01, 2019, 7:30 PM UTC

dkersten wrote: post #426

rdiddly wrote: post #343

Earlier quoted context omitted.

You've got an additional problem though, which is that this tells us you have two support channels: one that doesn't work (i.e. yours, the one you built), and one that does (Twitter-shaming). The first channel represents how you act when no one's watching; the second, how you act when they are. Most people prefer to deal with people for whom those two are the same.

As a DO user who was planning on ramping up usage in the coming weeks and months, this is what scares me and what is making me seriously reconsider.

https://github.com/fog/fog/issues/2525

https://news.ycombinator.com/item?id=6983097

Running anything business or privacy critical on DO is madness.

Re: "DigitalOcean Killed Our Company"

#582

Post by sneak » Sat, Jun 01, 2019, 7:31 PM UTC

jameshilliard wrote: post #445

bcooks wrote: post #270

As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…

Are you aware that Viasat has blacklisted a huge amount of Digitalocean /24 subnets? I can't access many of my servers when I'm on a satellite connection in addition to other websites hosted on Digitalocean. I've talked with the Viasat NOC and they told me they were blocking Digitalocean subnets due malware.

DO is famously bad at dealing with abuse reports; in a lot of cases they simply do nothing.

I block their netblocks for a lot of things, too.

Re: "DigitalOcean Killed Our Company"

#583

Post by dkersten » Sat, Jun 01, 2019, 8:48 PM UTC

xvector wrote: post #509

dkersten wrote: post #426

Earlier quoted context omitted.

As a DO user who was planning on ramping up usage in the coming weeks and months, this is what scares me and what is making me seriously reconsider.

Do not use DO. The very fact that their default response to suspected spam is to cause prod downtime is so bizarre and unacceptable that it does not make any sense whatsoever for a business to rely on them.

Thanks, I’ll stick with AWS then.

Re: "DigitalOcean Killed Our Company"

#584

Post by sn » Sat, Jun 01, 2019, 9:15 PM UTC

bluedino wrote: post #385

walrus01 wrote: post #151

Earlier quoted context omitted.

ISP here: The margins in bulk hosting services are incredibly thin, and companies have resorted to automation tools. If somebody asked me to run backend infrastructure for something like DigitalOcean or Linode, I would run away screaming. It would literally be my own personal hell. I would rather run any other sort of ISP services on the planet than a bulkhosting service where anybody with a pulse and $10 to $20/mont…

I like prgmr.com’s motto, “expect about $5/month in support”

srn from prgmr.com here. Our tagline originally came from us being a low cost service, but I like to think of it as a customer support philosophy. One meaning is we want you to be able to fix the problem yourself by giving you instructions instead of logging into your system. Another is we try give you the benefit of the doubt that it could be our problem and not assume it's yours when there's an issue.

Re: "DigitalOcean Killed Our Company"

#585

Post by nathan-io » Sat, Jun 01, 2019, 9:37 PM UTC

rat9988 wrote: post #571

nathan-io wrote: post #396

Earlier quoted context omitted.

I'm no longer able to edit the above comment, so to elaborate on #1: Following is a comment[1] by Moisey Uretsky in another thread[2]: > Depending on which items are flagged the account is put into a locked state, which means that access is limited. However, the droplets for that account and other services are not affected at all. The account is also notified about the action and a dialogue is opened, to determine wh…

What he said in an other thread, and this thread, is press release, marketing. Don't trust what he says to save his business. You have absolutely no reason to.

I prefer to give them the benefit of the doubt, though a clear explanation of why the above policy was not followed seems warranted. (It also doesn't appear to have been followed in several other instances reported by other former customers in various HN threads.)

If DO reserves the right to cut off services and access to your own data permanently and without warning (outside of a court order or confirmed illegal activity), that needs to be unequivocally stated, and the triggering factors should be made known. Otherwise, DO is not fit for production systems.

Additionally, it would be nice to see the creation of a transparent, high-level appeal process for customers affected by suspensions. Truly malicious customers wouldn't use it (what would they hope to successfully argue to an actual human reviewer?), but it would greatly benefit legitimate customers to have an outlet other than social media by which to "get something done" in the event of an inappropriate suspension followed by a breakdown in the standard review process.

Re: "DigitalOcean Killed Our Company"

#586

Post by bluedino » Sat, Jun 01, 2019, 10:13 PM UTC

sn wrote: post #584

bluedino wrote: post #385

Earlier quoted context omitted.

I like prgmr.com’s motto, “expect about $5/month in support”

srn from prgmr.com here. Our tagline originally came from us being a low cost service, but I like to think of it as a customer support philosophy. One meaning is we want you to be able to fix the problem yourself by giving you instructions instead of logging into your system. Another is we try give you the benefit of the doubt that it could be our problem and not assume it's yours when there's an issue.

which goes in line with “we don’t assume you’re stupid”

Re: "DigitalOcean Killed Our Company"

#587

Post by walrus01 » Sat, Jun 01, 2019, 10:26 PM UTC

cwyers wrote: post #553

walrus01 wrote: post #454

Earlier quoted context omitted.

Support and staffing costs are not subject to Moore's law.

Neither is bandwidth.

bandwidth does drop in price, and increase in capacity, at a fairly rapid rate. Look at what an ISP might pay for a 10GbE IP transit circuit in 2008 vs what you can get a 100GbE circuit for today. But peoples' bandwidth needs and traffic also grow rapidly.

Re: "DigitalOcean Killed Our Company"

#588

Post by hitekker » Sat, Jun 01, 2019, 11:35 PM UTC

ethbro wrote: post #549

zeta0134 wrote: post #335

Earlier quoted context omitted.

I'm fortunate that my own company (Rackspace) at least has a level head about this sort of thing. My direct manager looks at my numbers (~60-80 interactions per month) and my colleagues (many hundreds of interactions per month) and correctly observes that we have different strengths, and that's the end of the discussion. I have a tendency to take my time and go deep on issues, and my coworkers will send me tickets th…

Support should be looked at as a profit center, but almost everyone tries to run it like a cost center. It's crazy that companies spend $$ on marketing and sales, then cheap out on a interaction with someone who is already interested in / using their product.

Crazy from the individual perspective; natural from the group's.

"We have a mark, lets suck it dry until we can throw it away and find new and better marks."

Sustainability tends not be a concern until after a group's leadership jumps ship and parasitizes other hosts.

Re: "DigitalOcean Killed Our Company"

#589

Post by OBLIQUE_PILLAR » Sun, Jun 02, 2019, 4:09 AM UTC

Cakez0r wrote: post #482

chris_wot wrote: post #401

Earlier quoted context omitted.

It's not the false positive that is the issue here. The issue is that a. it took way too long to get the business back up and running, and b. the second response gave no explanation and no recourse for the business to become operational again. The very fact that this can happen from an automated script with no oversight should give every one of your customers pause as to whether they continue with your service.

I'd say the issue is that DO is shutting down servers for any reason at all (legal issues aside). If DO sells a product with a particular capacity, why should they intervene at all if a user is using all of the capacity they're paying for?

Gcp bans mining. Most providers frown on running tor exit nodes.

Re: "DigitalOcean Killed Our Company"

#590

Post by qmarchi » Sun, Jun 02, 2019, 5:16 AM UTC

xtracto wrote: post #570

9HZZRfNlpR wrote: post #460

Earlier quoted context omitted.

What do you recommend your clients to do if that kind of mistake happens to them? Is Twitter-shaming the only way out? I know people say some legal arguments why they close you down and won't say anything, but this is the worst scenario ever. I'd be better off excused at something I didn't do than just ooops we can't tell you anything, your account has been shut down.

This is important. I hate how it had became standard for companies to screw their customers unless they are online-shamed. The response email even read like a giant polite FUCK YOU (we locked your account, no further action required by you) You bet I will have further action! And it is after the shaming that you get an "I am sorry for this situation". Which sounds more like saying "I'm sorry we got caught". My frustr…

It's most likely due to the fact that the people who can act upon the process itself, not just follow the process inevitably see the issue and do truly want to help.

Getting your message into the right hands is what matters, not the platform it's on.