Live data from Hacker News

"DigitalOcean Killed Our Company"

twitter.com

351–360 of 620 posts

Re: "DigitalOcean Killed Our Company"

#351
post #135

Some people on HN hate Linode because of their past security screwups (which is valid), but having used both DO and Linode quite a lot, the support on Linode is way, way, way better than DO's. DO's tier 1 support is almost useless. I set up a new account with them recently for a droplet that needed to be well separated from the rest of my infrastructure, and ran into a confusing error message that was preventing it f…

I have interviewed with DO and they tried diverting me towards a support position. They told me that on a single day a support engineer was supposed to help/advice customers on pretty much whatever the customer was having issue with and also handle something between 80-120 tickets per day. It's nice to see that DO is willing to help on pretty much anything they (read: their team) has knowledge about, but with 80-120…

Yikes. I'd rather work manual labor vs hit 80-120/day ticket targets!

Re: "DigitalOcean Killed Our Company"

#352
post #347

Earlier quoted context omitted.

My understanding is that it isn't the 10 VMs as much as the resource usage (my suspicion is that DO is running a lot closer to the margin that larger providers, so they police this more). So they probably pegged all the CPUs at 100%. (perhaps a message queue approach would have been easier on the resources)

He did mention Redis, so I assume some sort of message queueing was in play.

Redis doesn't ensure it's being used with a message queue, but that's definitely a possibility.

Re: "DigitalOcean Killed Our Company"

#353
If you run mission-critical apps for F500 companies, you HAVE to have a backup/DR policy in place, using a different IT provider than your main one (and/or a different infrastructure site).

DO in my experience is not the greatest company to work with; but at the same time, your "incompetence" killed your company, not DO.

Re: "DigitalOcean Killed Our Company"

#354
Question : so what do you do besides host it yourself? Do you set up a backup on a different cloud service so that at least you can failover without too much downtime? you then have to at least pay for the some of the resources even if there is no traffic. or backup locally but have a process set up with the different service so you can provision and get back up quickly? or is there a better solution?

Re: "DigitalOcean Killed Our Company"

#355
post #250

Earlier quoted context omitted.

Accidents can happen. Don't really blame Digital Ocean for the accidental locking but this response is insane: https://pbs.twimg.com/media/D76ocofXoAY_xB5.png

DO has shown that their service is simply not suitable for some use cases: those that impose an "unreasonable" load on their infraestructure. Even worse: they don't explicitly state what is considered "unreasonable". So, if your business is serious, you have to assume the worst-case scenario: DO can't be used for anything Conclusion: Digital Ocean is just for testing, playing around, not suitable for production.

[deleted]

Re: "DigitalOcean Killed Our Company"

#356
post #250

Earlier quoted context omitted.

Accidents can happen. Don't really blame Digital Ocean for the accidental locking but this response is insane: https://pbs.twimg.com/media/D76ocofXoAY_xB5.png

DO has shown that their service is simply not suitable for some use cases: those that impose an "unreasonable" load on their infraestructure. Even worse: they don't explicitly state what is considered "unreasonable". So, if your business is serious, you have to assume the worst-case scenario: DO can't be used for anything Conclusion: Digital Ocean is just for testing, playing around, not suitable for production.

> Conclusion: Digital Ocean is just for testing, playing around, not suitable for production.

I think that's always been the standard position most people take. DO, Linode, etc are for personal side projects, hosting community websites, forums etc. They are not for running a real business on. Some people do, sure but if hosting cost is really that big a portion of your total budget you probably don't have a real business model yet anyway.

Re: "DigitalOcean Killed Our Company"

#357
post #270

As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…

Will the customer be compensated for business losses?

Re: "DigitalOcean Killed Our Company"

#358

Let me see if I get this straight... Developer has a Python script that takes 1 second per record to execute and he has 500,00 records to process, so he spins up 10 distinct VMs each running the same Python script to parallelize the task. The provider shuts him down and cites a section of the EULA that says "You shall not take any action that imposes an unreasonable... load on our infrastructure." Basically saying "H…

Except it wasn’t “whatever you did, don’t do that.” It was, “this looks weird, can you explain?” Followed by “OK, sounds good, carry on.” Then they got shut down.

Re: "DigitalOcean Killed Our Company"

#359
post #270

As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…

You've got an additional problem though, which is that this tells us you have two support channels: one that doesn't work (i.e. yours, the one you built), and one that does (Twitter-shaming). The first channel represents how you act when no one's watching; the second, how you act when they are. Most people prefer to deal with people for whom those two are the same.

As someone who has been blown off by DO support, you hit the nail on the head.

Re: "DigitalOcean Killed Our Company"

#360

Some people on HN hate Linode because of their past security screwups (which is valid), but having used both DO and Linode quite a lot, the support on Linode is way, way, way better than DO's. DO's tier 1 support is almost useless. I set up a new account with them recently for a droplet that needed to be well separated from the rest of my infrastructure, and ran into a confusing error message that was preventing it f…

Does DO have different levels of support that you can pay for like AWS? I like that system. You pay when you need it. You pay more if you need more support.

The difference in support DO vs Linode is probably due to DO being cheaper.

Post reply on HN