Live data from Hacker News

DigitalOcean lost our server

murze.be

181–190 of 219 posts

Re: DigitalOcean lost our server

#183
post #152

I'm more worried about the following post in that thread: -------------------------------------------------------- We were loyal customers of DigitalOcean for over 2 years. We showed up to work one morning and had a client email stating that their website was down. We checked, sure enough. We tried logging in to our DI account and it said it was suspended. After searching around, we realized our CC had expired. No bi…

>We've been a customer for 2 years, surely they could pick up the phone and call us.

I don't understand that reasoning from customers to be honest. That right there, throws away the $5 and $10 plans. Having some one trying to contact a customer, just because their credit card expired, cost more than the actual hosting.

Top tier service does comes from companies with $5 a month plans. I'm sorry it just doesn't.

Re: DigitalOcean lost our server

#184
post #147
post #76

Earlier quoted context omitted.

> You should be designing for failure in "the cloud" DO should be designing for failure: VM storage should be on a SAN. A single physical server failing should not cause data loss. This is basic stuff.

SANs aren't magical boxes incapable of losing data.

I worked for a company that believed that, for a while.

They bought a hugely expensive SAN solution (HP I think). One of my questions was: What happens when the SAN fails? Well you see, because it has redundancy, that can't happen. Clever uh?.

First time it failed, everything gridded to a halt for two days. The second time they did better and had it running within eight hours.

Re: DigitalOcean lost our server

#185
post #147
post #76

Earlier quoted context omitted.

> You should be designing for failure in "the cloud" DO should be designing for failure: VM storage should be on a SAN. A single physical server failing should not cause data loss. This is basic stuff.

SANs aren't magical boxes incapable of losing data.

Nobody claimed that.

Re: DigitalOcean lost our server

#186

This seems like a non-issue to me. If you're using an IaaS provider you should be treating the network as volatile from the get-go. This is the reason AWS has things like auto-scaling groups. You should be designing for failure in "the cloud"

While my first intuition was to agree with you, there's certainly an upcoming generation of developers who have never operated their own root servers and the abstraction level in the cloud nowadays is so high it makes you easily forget that "Droplets" are just VMs are just servers running software. On the other hand, hardware reliability has increased in recent years due to RAID, fully redundant networking & power ad…

This seems like the wrong lesson is being taught though somewhere - the delete button is right there. It should be much more obvious that is very easy to make a server go away.

Re: DigitalOcean lost our server

#187
post #152

I'm more worried about the following post in that thread: -------------------------------------------------------- We were loyal customers of DigitalOcean for over 2 years. We showed up to work one morning and had a client email stating that their website was down. We checked, sure enough. We tried logging in to our DI account and it said it was suspended. After searching around, we realized our CC had expired. No bi…

This is a big danger of "move fast and break things". I imagine the process they put in place seemed reasonable internally before catching this edge case, but what a crappy result.

Re: DigitalOcean lost our server

#188
post #152

I'm more worried about the following post in that thread: -------------------------------------------------------- We were loyal customers of DigitalOcean for over 2 years. We showed up to work one morning and had a client email stating that their website was down. We checked, sure enough. We tried logging in to our DI account and it said it was suspended. After searching around, we realized our CC had expired. No bi…

Thank you for this, I have a large client where we were considering DigitalOcean and the yearly spend could easily be in the thousands. Now I'll strike them off of the list. Automated deletion of VM's and backups, absolutely unacceptable.

The VM hosting market isn't looking good at the moment, seems like each provider has some dark history at this point, it's really frustrating.

Re: DigitalOcean lost our server

#189
post #152

I'm more worried about the following post in that thread: -------------------------------------------------------- We were loyal customers of DigitalOcean for over 2 years. We showed up to work one morning and had a client email stating that their website was down. We checked, sure enough. We tried logging in to our DI account and it said it was suspended. After searching around, we realized our CC had expired. No bi…

>We've been a customer for 2 years, surely they could pick up the phone and call us. I don't understand that reasoning from customers to be honest. That right there, throws away the $5 and $10 plans. Having some one trying to contact a customer, just because their credit card expired, cost more than the actual hosting. Top tier service does comes from companies with $5 a month plans. I'm sorry it just doesn't.

> We've spent thousands of dollars with them.

That's a decent amount of money right? It sounds like they were not using the $5 plans.

Re: DigitalOcean lost our server

#190
post #152

I'm more worried about the following post in that thread: -------------------------------------------------------- We were loyal customers of DigitalOcean for over 2 years. We showed up to work one morning and had a client email stating that their website was down. We checked, sure enough. We tried logging in to our DI account and it said it was suspended. After searching around, we realized our CC had expired. No bi…

Offsite backups? We use AWS and DO to host applications for our clients. We still take offsite backups. Isn't that what your clients paying you for?
Post reply on HN