Earlier quoted context omitted.
SANs aren't magical boxes incapable of losing data.
Nobody claimed that.
Yet you suggest DO should be using SANs when there isn't enough information provided to know if it would have made a difference in the outcome.
DigitalOcean lost our server
201–210 of 219 posts
Re: DigitalOcean lost our server
#202We are also experiencing a huge problem with performance of DO servers as we scale. Especially databases. With stories like that , a decision of moving to AWS seems pretty obvious.
Re: DigitalOcean lost our server
#203Probably due to optimization of the conversion funnel, it's VERY easy to build things on DigitalOcean without understanding that - unlike many other *-as-a-service - droplets do not include backups and that is your responsibility. As a sysadmin, this is fine since I don't trust a single provider for both prod and backups anyhow, but I fear for those who are just getting started and don't realize this. I agree that DO…
I would really hope that the presence of a "Backups" checkbox on the droplet creation page is enough to tell the (presumably tech-savvy) user that backups don't happen unless you check that box.
I still don't trust them to back it up. I checked the box, still make my own local backup.
Re: DigitalOcean lost our server
#204Earlier quoted context omitted.
I would really hope that the presence of a "Backups" checkbox on the droplet creation page is enough to tell the (presumably tech-savvy) user that backups don't happen unless you check that box.
I still don't trust them to back it up. I checked the box, still make my own local backup.
That's probably wise, I really should do the same. My important stuff is in git with a remote repository, but losing all my configuration and setup would still not be great.
Re: DigitalOcean lost our server
#205Earlier quoted context omitted.
Nobody claimed that.
Yet you suggest DO should be using SANs when there isn't enough information provided to know if it would have made a difference in the outcome.
I suggest Digital Ocean should be using any mechanism that stops a single server from affecting data. A SAN backed block store allows you to provide rules for the level of redundancy required, as well as replication mechanisms for things like firmware failures, to a much greater degree than a host backed RAID array.
That's completely different from suggesting 'SANs are magical boxes incapable of losing data'. Have you considered apologising?
Re: DigitalOcean lost our server
#206Earlier quoted context omitted.
Yet you suggest DO should be using SANs when there isn't enough information provided to know if it would have made a difference in the outcome.
I suggest Digital Ocean should be using any mechanism that stops a single server from affecting data. A SAN backed block store allows you to provide rules for the level of redundancy required, as well as replication mechanisms for things like firmware failures, to a much greater degree than a host backed RAID array. That's completely different from suggesting 'SANs are magical boxes incapable of losing data'. Have yo…
Servers affect data. That's kind of the whole point of servers. Maybe a multi-SAN setup with synchronous replication would have prevented the data loss. Maybe not. Not enough information has been provided to know if the cause of the data loss was the storage or the server.
But if DO had a multi-SAN setup with synchronous replication this thread wouldn't exist because DO's business model of "super cheap VPS w/ fast SSD storage" would have failed due to costs. Everybody wants Five Nines until they have to pay for it...
Re: DigitalOcean lost our server
#207Earlier quoted context omitted.
I suggest Digital Ocean should be using any mechanism that stops a single server from affecting data. A SAN backed block store allows you to provide rules for the level of redundancy required, as well as replication mechanisms for things like firmware failures, to a much greater degree than a host backed RAID array. That's completely different from suggesting 'SANs are magical boxes incapable of losing data'. Have yo…
Servers affect data. That's kind of the whole point of servers. Maybe a multi-SAN setup with synchronous replication would have prevented the data loss. Maybe not. Not enough information has been provided to know if the cause of the data loss was the storage or the server . But if DO had a multi-SAN setup with synchronous replication this thread wouldn't exist because DO's business model of "super cheap VPS w/ fast S…
Digital Ocean sell 'Simple Cloud Infrastructure'. Server data might be transient, but most web developers aren't infrastructure engineers and will have no idea of this - certainly it seems to have come as a surprise to some people. Digital Ocean need to manage their customer's expectations - and upsell for people who want permanent storage - as a rude shock doesn't do well for their brand.
Re: DigitalOcean lost our server
#208I'm more worried about the following post in that thread: -------------------------------------------------------- We were loyal customers of DigitalOcean for over 2 years. We showed up to work one morning and had a client email stating that their website was down. We checked, sure enough. We tried logging in to our DI account and it said it was suspended. After searching around, we realized our CC had expired. No bi…
This is why you use two providers with two credit cards.
I'm quite serious. Our production environment operates on two cards with different issue & expiration dates from two different merchants and uses 2 different Cloud providers.
Re: DigitalOcean lost our server
#209Earlier quoted context omitted.
Anecdotally: Every company I've worked for that uses hardware raid controllers, it feels like the controllers break about as often as one of the disks. Sure, still an improvement over no RAID, but still ridiculous compared to any decent software raid (md or ZFS).
You lose a RAID card as often as spinning metal disks? Sure that's not a wild exaggeration? Sure sounds like one. In 15+ years of doing this, I've only been a part of a RAID card swap three times. Once was just to be safe with an overly paranoid customer. That said, I'm a fan of moving away from RAID. It's just more complexity in the face of a movement towards simpler architectures. The one benefit of a good RAID car…
It may be. As I said, "it feels like" - I'm probably assigning some selective memory bias because when it does happen it's much more of a pain.
Re: DigitalOcean lost our server
#210Earlier quoted context omitted.
Yep, somehow cloud customers have decided that by being in the cloud everything is redundant and durable and has several other magical properties. At one of my positions we had stupidly put too many eggs in the basket of a single physical machine. Its disk controller failed in a way that it trashed the data volumes. I was unable to convince anyone that "move to Amazon" was not a one-step solution to "how do we make s…
>somehow cloud customers have decided that by being in the cloud everything is redundant and durable and has several other magical properties To be fair the point of the cloud is that they deal with redundancy, HA, distributing to multiple datacenters, etc. through their services - but you need to use and understand implications of said services to leverage that.
When cloud == SaaS, sometimes.
When cloud == IaaS, nope.
But that doesn't stop the misconception.