Live data from Hacker News

"DigitalOcean Killed Our Company"

twitter.com

451–460 of 620 posts

Re: "DigitalOcean Killed Our Company"

#451
post #48
post #19

Earlier quoted context omitted.

There's a pretty hard cap on the level of redundancy you can do with a two-man company, as I assume a two-man company does not bring in a lot of money.

Agreed but having some backups on a different platform and a fallback plan for deploying somewhere else should be doable.

Are there guides to this that take you through good configs for smaller environments like this? i.e. you have a postgres DB and two web servers. What's the simplest backup process and how do you replicate to another DB VM over on another VPS provider securely?

Stuff like this is relatively simple when you are trying to learn it, but keeping it operation is hard at a small level. Is the less to really use PaaS until it's viable for you to be running a small K8s cluster, or equivalent fleet? Seems really expensive compared to VPSes, but having better guides might help.

Re: "DigitalOcean Killed Our Company"

#452

Bookmarking this for next time there's a HN comment "why is Netflix using AWS, they should just get some cheap VPS's from DO" The rise of the public cloud is pretty fortunate for companies like DO, they can make the same assumptions that legacy VPS companies did (most vm's sit idle so oversell them massively, most customers will create a single vm and nothing else) while branding themselves as having the same strong…

Has anyone ever actually said Netflix should switch from AWS to "some cheap VPS's from DO"? AWS offers a lot that DO doesn't. They're hardly even comparable.

Re: "DigitalOcean Killed Our Company"

#453
post #441

Some people on HN hate Linode because of their past security screwups (which is valid), but having used both DO and Linode quite a lot, the support on Linode is way, way, way better than DO's. DO's tier 1 support is almost useless. I set up a new account with them recently for a droplet that needed to be well separated from the rest of my infrastructure, and ran into a confusing error message that was preventing it f…

Vultr is underrated too. I've had nothing but positive tech support experiences with them. Their weird branding turns people off but they do not seem fly by night. We have used them for various things for years with very few problems.

My only issue is the provisioning weirdness.

I've had interesting issues automating deployment to the point that my current build script provisions 9 VMs, benchmarks them and shuts down the worst performing 6. Some of their co-location is CPU stressed.

Re: "DigitalOcean Killed Our Company"

#454

Earlier quoted context omitted.

Personal opinion, it's really important in the ISP/hosting world to identify what market categories are a race to the bottom, and if at all possible, refuse to participate in them. I look at companies selling $5 to $15/month VPS services and try to figure out how many customers they need to be set up for monthly recurring services, in order to pay for reasonably reliable and redundant infrastructure, and the math jus…

Ten years ago, I had a pretty reasonable $10/mo account, that I eventually moved to a $20/mo account because I needed more resources to keep up with traffic. I'd expect that, as more transistors have been packed blah blah blah, that such a $10/mo account would have gotten better , not worse, since then.

Support and staffing costs are not subject to Moore's law.

Re: "DigitalOcean Killed Our Company"

#455
post #178

Earlier quoted context omitted.

And Linode has always had faster CPU, I/O, Network. And lots of small things like that DO only catches up in the recent years, like pooled bandwidth. Although these days I tend to go with UpCloud, it is very similar to Linode and DO, except you can do custom instances like 20x vCPU with 1GB Memory, spin it up for $0.23 an hour. Compared to standard plan on Linode and DO, 20 vCPU with 96GB Memory would be $0.72/Hr.

I'd love to see the benchmarks you used. I've done a bunch over the years, mostly I/O focused because that's the most common bottleneck for the kind of work I do. While Linode does pretty well, especially compared to the cloud giants, DO has pretty much always come out on top.

Really depends on underlying hardware of the Linode and DO VPS servers as they can vary greatly especially on DO depending on datacenter and region you end up in. Newer DO datacenters getting newer hardware so difference compared to older DO datacenters is huge - benchmarks of same DO droplet plan on different hardware https://community.centminmod.com/threads/digitalocean-us-15-...

Re: "DigitalOcean Killed Our Company"

#456
post #250

Earlier quoted context omitted.

Accidents can happen. Don't really blame Digital Ocean for the accidental locking but this response is insane: https://pbs.twimg.com/media/D76ocofXoAY_xB5.png

> Don't really blame Digital Ocean for the accidental locking I find it concerning that they have such a low threshold for triggering a lock. 10 droplets is hardly cloud scale.

Used to work at Linode, let's flip this on it's head:

-When the majority of abuse support dealt with was people angrily calling and asking about the fraudulent charges on their cards for dozens of Lie-nodes you consider putting caps in place to reduce support burden and reduce chargebacks.

At the time at Linode, if it was a known customer, we could easily and quickly raise that limit and life is good.

I've always wondered how Amazon dealt with fraud/abuse at their scale.

I don't think DO was wrong here to have a lock, but the post lock procedure seemed to be the problem.

Re: "DigitalOcean Killed Our Company"

#457

Some people on HN hate Linode because of their past security screwups (which is valid), but having used both DO and Linode quite a lot, the support on Linode is way, way, way better than DO's. DO's tier 1 support is almost useless. I set up a new account with them recently for a droplet that needed to be well separated from the rest of my infrastructure, and ran into a confusing error message that was preventing it f…

My current employer uses Linode, and yeah, they have pretty good support. However, I've been using DO for the last 5 years, and haven't needed to reach out to support once. But I've had to contact Linode support about 5 times in the last year.

Re: "DigitalOcean Killed Our Company"

#458
post #368
post #270

As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…

Thanks for the replies. Let me try to address a few of the things I have seen here. We haven't completed our investigation yet which will include details on the timeline, decisions made by our systems, our people, and our plans to address where we fell short. That said, I want to provide some information now rather than waiting for our full post-mortem analysis. A combination of factors, not just the usage patterns,…

I have always been a happy DO customer. So glad to see you step into this lions den and reply to the community.

Looking forward to the write up!

Re: "DigitalOcean Killed Our Company"

#459

Earlier quoted context omitted.

I once worked for eBay a long time ago, and support consisted of 4 concurrent chats, offering pre-programmed macros often pointing to terribly written documentation the person had already read and was confused about. If you took the time to actually assist somebody you were chastised in a weekly review where they went over your chat support. The person doing mine told me I had the highest satisfaction record in the e…

I'm fortunate that my own company (Rackspace) at least has a level head about this sort of thing. My direct manager looks at my numbers (~60-80 interactions per month) and my colleagues (many hundreds of interactions per month) and correctly observes that we have different strengths, and that's the end of the discussion. I have a tendency to take my time and go deep on issues, and my coworkers will send me tickets th…

+1 for Rack support. A previous company I worked for was heavily invested in Rackspace infrastructure and while I often opined not getting the equivalent experience with AWS for the resume, I was regularly floored with the quality of their support. Whenever I had the pleasure of needing to open a ticket they solved my problems and usually taught me something new in the process. The linux guys were very clearly battle hardened admins.

I couldn't imagine getting that level of support from DO, let alone Amazon.

Re: "DigitalOcean Killed Our Company"

#460
post #368
post #270

As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…

Thanks for the replies. Let me try to address a few of the things I have seen here. We haven't completed our investigation yet which will include details on the timeline, decisions made by our systems, our people, and our plans to address where we fell short. That said, I want to provide some information now rather than waiting for our full post-mortem analysis. A combination of factors, not just the usage patterns,…

What do you recommend your clients to do if that kind of mistake happens to them? Is Twitter-shaming the only way out?

I know people say some legal arguments why they close you down and won't say anything, but this is the worst scenario ever. I'd be better off excused at something I didn't do than just ooops we can't tell you anything, your account has been shut down.

Post reply on HN