Wow. well, I will avoid DO at all costs for the rest of my career. There is a reason Enterprises trust AWS, GCP, Azure, and pay for premium support (and startups should, to the best of their financial abilities). He needs to sue or negotiate to recover material damages caused by this.
"DigitalOcean Killed Our Company"
431–440 of 620 posts
Re: "DigitalOcean Killed Our Company"
#432I am a software developer and yes software is eating the world. One of the side effects of software eating the world is out of control software. * Autobans in facebook * Cheated instacart drivers * $10000 stolen from thousands of bank accounts (and returned hopefully) on Etsy * Tesla cars literally killing people (it now feels like it's once a month right?) Now the software that runs software is running amok. The int…
Re: "DigitalOcean Killed Our Company"
#433Earlier quoted context omitted.
Personal opinion, it's really important in the ISP/hosting world to identify what market categories are a race to the bottom, and if at all possible, refuse to participate in them. I look at companies selling $5 to $15/month VPS services and try to figure out how many customers they need to be set up for monthly recurring services, in order to pay for reasonably reliable and redundant infrastructure, and the math jus…
"you're responsible for doing all your own offsite backups" That's going to be true no matter which cloud provider you choose. Their ToS almost certainly include terms which allow them to kick you off and refund any monies for any reason whatsoever. Good luck if you bought into their entire ecosystem and can't move elsewhere on a whim.
My company provides services to fortune 100 companies, and we host literally petabytes of data on their behalf in Amazon S3, but we don't have offsite backups. We (and they) rely on Amazon's durability promise.
We do offer the option of replicating their data to another cloud provider, but few customers use that service -- few companies want to pay over twice the cost of storage for a backup they should never need to use when the provider promises 99.999999999% durability.
Re: "DigitalOcean Killed Our Company"
#434DigitalOcean is operating worse than a fly by night host (like AlphaRacks, GreenValueHost, etc). The reasonable course of action would've been to email the customer and throttle their API access to prevent load spikes, but DO instead locked their entire account (not just the service that DO felt was being abused). A fly by night will often only suspend the VM or database that is in question, not other services on the…
> worse than a fly by night host (like AlphaRacks I saw a great deal on LEB for a KVM VPS from Alpharacks and signed up for a 2 year plan (my first mistake). When SSHing in to the VPS it didn't have the advertised specs, and when I raised the issue with their support, they eventually fixed it.. Then I realized the second problem, they gave me the same IP address as someone else. You could still use the web VNC consol…
Re: "DigitalOcean Killed Our Company"
#435Earlier quoted context omitted.
You said "not in my experience they don't" and then literally describe in detail how they did exactly what I was saying they do.
I guess he meant they don't lock your whole account but only stop a single server.
Re: "DigitalOcean Killed Our Company"
#436Earlier quoted context omitted.
I have interviewed with DO and they tried diverting me towards a support position. They told me that on a single day a support engineer was supposed to help/advice customers on pretty much whatever the customer was having issue with and also handle something between 80-120 tickets per day. It's nice to see that DO is willing to help on pretty much anything they (read: their team) has knowledge about, but with 80-120…
This is appalling. I worked as a L1 ticket tech for an old LAMP host back in the day where probably half of the tickets required nothing more than a password reset or a IP removal from our firewall, very easy stuff, and was proud if I got over 60 responses out in a 8 hour shift. And that time was spent mostly just typing a response to the customer. I really expected higher standards out of DO.
Re: "DigitalOcean Killed Our Company"
#437Earlier quoted context omitted.
Long-time Linode customer here. What were the security screwups? I tried a couple searches but didn't turn up anything.
The two major ones were: A Bitcoin theft via the Linode Manager interface in 2012: https://news.ycombinator.com/item?id=3654110 and a second Linode Manager compromise in 2016: https://news.ycombinator.com/item?id=10845170 In both cases, if you dig into the context a bit, the story turned out that Linode wasn't fully disclosing the breaches to their customers until they were forced to do so when the news about them re…
This is entirely the roots of my distrust of them right now. Mistakes happen. Companies I trust demonstrate that they've learned from their mistakes. My tolerance for mistakes is pretty low when it comes to security related things, though. If something has gone wrong, let me know so I can take remedial steps. Their handling of both of those incidents suggested I can't trust them to tell me in sufficient time to protect myself.
Re: "DigitalOcean Killed Our Company"
#438Some people on HN hate Linode because of their past security screwups (which is valid), but having used both DO and Linode quite a lot, the support on Linode is way, way, way better than DO's. DO's tier 1 support is almost useless. I set up a new account with them recently for a droplet that needed to be well separated from the rest of my infrastructure, and ran into a confusing error message that was preventing it f…
And Linode has always had faster CPU, I/O, Network. And lots of small things like that DO only catches up in the recent years, like pooled bandwidth. Although these days I tend to go with UpCloud, it is very similar to Linode and DO, except you can do custom instances like 20x vCPU with 1GB Memory, spin it up for $0.23 an hour. Compared to standard plan on Linode and DO, 20 vCPU with 96GB Memory would be $0.72/Hr.
Re: "DigitalOcean Killed Our Company"
#439Some people on HN hate Linode because of their past security screwups (which is valid), but having used both DO and Linode quite a lot, the support on Linode is way, way, way better than DO's. DO's tier 1 support is almost useless. I set up a new account with them recently for a droplet that needed to be well separated from the rest of my infrastructure, and ran into a confusing error message that was preventing it f…
Does DO have different levels of support that you can pay for like AWS? I like that system. You pay when you need it. You pay more if you need more support. The difference in support DO vs Linode is probably due to DO being cheaper.
What? Most DO and Linode plans have exact same specs and cost exactly the same, and IIRC it was DO matching Linode. Although DO didn't seem to enforce their egress budget while Linode does.
(I've been a customer of both for many years, and only dropped DO a few months ago.)
EDIT: Also, according to some benchmarks (IIRC), for the exact same specs Linode usually has an edge in performance.
Re: "DigitalOcean Killed Our Company"
#440As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…