Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"DigitalOcean Killed Our Company"

twitter.com

Post reply on HN
Print view View original

591–600 of 620 posts

Re: "DigitalOcean Killed Our Company"

#591

Post by fastball »

fgonzag wrote: post #306
fastball wrote: post #279

Earlier quoted context omitted.

That's probably a reason to use containerization / other technologies so that you can spin up your services in a couple minutes on a different cloud provider.

You don't need to use containers for that.. all you have to do is set up a warm replica of the service with another provider. The fail over doesn't even have to be automatic, but that is the minimum amount of redundancy any production SaaS should have.

A "warm replica" is going to cost money though, while containerization allows you to not have anything spun up until the moment you need it, and then have it ready to go minutes / an hour later.

Re: "DigitalOcean Killed Our Company"

#592

Post by Phlarp »

bluedino wrote: post #555
Phlarp wrote: post #459

Earlier quoted context omitted.

+1 for Rack support. A previous company I worked for was heavily invested in Rackspace infrastructure and while I often opined not getting the equivalent experience with AWS for the resume, I was regularly floored with the quality of their support. Whenever I had the pleasure of needing to open a ticket they solved my problems and usually taught me something new in the process. The linux guys were very clearly battle…

i have the opposite experience with Rackspace. The low end stuff (hosted exchange etc) is basically useless, people who are obviously on multiple chats, they let tickets sit for days... Even when we had small handful of physical servers with them, they seemed inept. They actually lost our servers one time and couldn’t get someone out to reset power on our firewall.

My experiences were all with their "dedicated" or "managed" cloud services. Although I did notice that their marketing seemed to shift in the last months I was working with them for that employer from "let us help you build things on Rackspace" to "let us help you move what you built on Rackspace to AWS"

Re: "DigitalOcean Killed Our Company"

#593

Post by fgonzag »

fastball wrote: post #591
fgonzag wrote: post #306

Earlier quoted context omitted.

You don't need to use containers for that.. all you have to do is set up a warm replica of the service with another provider. The fail over doesn't even have to be automatic, but that is the minimum amount of redundancy any production SaaS should have.

A "warm replica" is going to cost money though, while containerization allows you to not have anything spun up until the moment you need it, and then have it ready to go minutes / an hour later.

That is patently false, unless you plan on starting from a clean slate on the new environment. Any one who proposed such a solution as a business continuity practice to me would be immediately fired.

Containers solve the easy problem, which is how to make sure the dev environment matches the production environment. That is it.

Replicating TBs worth of data and making sure the replica is relatively up to date is the hard part. So is fail over and fail back. Basically everything but running the code/service/app, which is the part containers solve.

Re: "DigitalOcean Killed Our Company"

#594

Post by fastball »

fgonzag wrote: post #593
fastball wrote: post #591

Earlier quoted context omitted.

A "warm replica" is going to cost money though, while containerization allows you to not have anything spun up until the moment you need it, and then have it ready to go minutes / an hour later.

That is patently false, unless you plan on starting from a clean slate on the new environment. Any one who proposed such a solution as a business continuity practice to me would be immediately fired. Containers solve the easy problem, which is how to make sure the dev environment matches the production environment. That is it. Replicating TBs worth of data and making sure the replica is relatively up to date is the h…

I was responding to this comment:

> Sure, a backup would have been a significant improvement, but still – a backup only protects against data loss and not against downtime.

Assuming you have data backup / recovery good to go, the downtime issue needs to be solved by getting your actual web application / logic up and running again. With something like docker-compose, you can do this on practically any provider with a couple of commands. Frontend, backend, load-balancer -- you name it, all in one command.

> Containers solve the easy problem, which is how to make sure the dev environment matches the production environment. That is it.

Speaking of "patently false"...

Re: "DigitalOcean Killed Our Company"

#595

Post by zeta0134 »

Phlarp wrote: post #592
bluedino wrote: post #555

Earlier quoted context omitted.

i have the opposite experience with Rackspace. The low end stuff (hosted exchange etc) is basically useless, people who are obviously on multiple chats, they let tickets sit for days... Even when we had small handful of physical servers with them, they seemed inept. They actually lost our servers one time and couldn’t get someone out to reset power on our firewall.

My experiences were all with their "dedicated" or "managed" cloud services. Although I did notice that their marketing seemed to shift in the last months I was working with them for that employer from "let us help you build things on Rackspace" to "let us help you move what you built on Rackspace to AWS"

Yes, the Public Cloud, which houses most of the smaller Managed Infrastructure accounts (minimal support) is one of the bigger ... I believe the polite word is "opportunities?" It's a very pretty UI on top of a somewhat fragile Open Stack deployment, which needs a significant amount of work to patch around noisy infrastructure problems. That turns into a support floor burden, and it shows in ticket latency. Critiques directed at that particular product suite are, frankly, quite valid. I think Rackspace tried to compete with AWS, realized very quickly that they do not have Amazon's ability to rapidly scale, and very nearly collapsed under their own weight.

That said, our FAWS team are a good bunch, and what AWS lacks in support they more than make up for in well engineered, stable infrastructure. Since Rackspace's whole focus is support, I think the pairing works well on paper and it should scale effectively, but we'll have to see how it plays out in practice.

Re: "DigitalOcean Killed Our Company"

#598

Post by justforyou »

bcooks wrote: post #270

As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…

Do you realize that by abusing this thread to make a single PR focused comment with no intention of participating in the conversation -- you've disrespected the community here and the few remaining DO customers within said community.

Re: "DigitalOcean Killed Our Company"

#599

Post by mercer »

justforyou wrote: post #543
bcooks wrote: post #270

As DigitalOcean's CTO, I'm very sorry for this situation and how it was handled. The account is now fully restored and we are doing an investigation of the incident. We are planning to post a public postmortem to provide full transparency for our customers and the community. This situation occurred due to false positives triggered by our internal fraud and abuse systems. While these situations are rare, they do happe…

>> and we take every effort to get customers back online as quickly as possible. In this particular scenario, we were slow to respond and had missteps in handling the false positive. You clearly don't make every effort, and did not -- so why waste the extra verbiage and switch from active to passive voice? Based on your cliche response I have zero confidence that DO will do anything substantial to address the root ca…

I've found DO's public posts to be particularly grating in the "we are listening to YOU, our customer. we take feedback extremely seriously" department.

Re: "DigitalOcean Killed Our Company"

#600

Post by geezerjay »

thanatos_dem wrote: post #463
iamaelephant wrote: post #449

Earlier quoted context omitted.

I think it says a lot that this CTO joker flew in, regurgitated the standard-issue "we will endeavor to do better" apology and left without answering any of the very legitimate follow-up questions. I would never deal with an organisation that behaves like these guys.

Is there any response that would satisfy you?

> Is there any response that would satisfy you?

Do you believe that a PR response made in damage control mode that actually changes nothing is something that's satisfactory?

I mean, apparently this screwup was so damaging that it killed a company. What part of the PR statement addresses that precendent?

Post reply on HN