Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Highly Available Block Storage

digitalocean.com

Post reply on HN
Print view View original

51–60 of 145 posts

Re: Highly Available Block Storage

#51

Post by velodrome »

brianwawok wrote: post #33
johnwheeler wrote: post #31

I love how DO focuses on what matters the most: Inexpensive VMs and scalable block storage. If I had to pick two, those would be them!

Well except being 4 years late to the block storage game? Seems they have an uphill fight. Aws and gce match them on low end droplet price and offer much more. No local ssd but not sure what % of apps really need local ssd. DO effectively forces you to pay for local ssd for all of your servers.

Well, they beat Linode..

With DO, at least there is an additional option available for users.

With GCE and AWS, the outbound bandwidth is expensive. 1TB = $90 (AWS, GCE) vs 1TB included (Linode, DO).

Re: Highly Available Block Storage

#52

Post by andybak »

Jedd wrote: post #37
andybak wrote: post #18

This helps me with a nicer deployment setup. I was always keen on 'rebuild from scratch' rather than 'update stuff and hope you're idempotent and have captured all changes' but transitory data was always the problem. Now I can start building a new updated droplet and the only downtime will be that needed to detach and reattach the block storage containing the db etc. Anyone see a flaw in this? (I know there are other…

> Anyone see a flaw in this? Perhaps not a flaw, but some issues with your setup are implied. If you're rebuilding from scratch because you're not sure that you can update things, then you're probably in need of a configuration management tool (I'm a big fan of saltstack[1], mostly because I don't like Ruby or DSL's, but there's lots of options out there[2]) If you're worried you're going to lose transitory data, it…

I probably wasn't clear:

1. I am already using Ansible - but I want to run my "build from scratch" playbook every time I push a change instead of my "git pull etc." playbook.

2. I'm not scared of losing data because of backups or similar - it's because I want to start from a clean droplet every time I push a change.

I do know in theory that playbooks are idempotent and running the 'build from scratch' playbook on a running instance should result in the same state as a genuine 'build from scratch' but there are many places this can fail:

1. Someone has inadvertently modified the environment at some point. A quick-fix or accidental change or an alteration in my playbook that isn't clearly reflected in the running instance

2. Not all ansible modules are perfectly idempotent. It's a fairly leaky abstraction.

3. If you never build from scratch apart from the first time then you're not really testing your automated deployment. The next time you really need to build from scratch you might get a shock.

Re: Highly Available Block Storage

#53

Post by Someone1234 »

corobo wrote: post #45
bjacobel wrote: post #34

Reminder that just a few weeks ago DigitalOcean rolled over on one of their customers and took down 38,000 websites after receiving a claim of infringement from the NRA against a parody site hosted on surge.sh: http://motherboard.vice.com/read/nra-complaint-takes-down-38...

Reminder that you have to act on abuse notifications sharpish. You're providing a service, it's on you if you ignore abuse notifies. "We received notice on behalf of a trademark holder that a customer of DigitalOcean was hosting infringing content on our network. DigitalOcean immediately notified our customer of the infringement, and the customer was given a five day period to resolve the issue. The infringing conten…

That statement is problematic, namely this:

> The infringing content was not removed within the specified period even though several notifications were issued.

You don't have to remove content under the DMCA, you can also file a counter-notice which gets the content host off the hook and then the matter goes to court[0].

But that also assumes DMCA which, if memory serves, was not in play here. It was a trademark complaint, which DigitalOcean has no responsibility to resolve.

Ultimately DigitalOcean's response, even with that statement, seems at odds with how the law is actually written. The other party also claimed they did respond to DigitalOcean, they just never removed the legal parody material which is their right.

DigitalOcean's understanding of the NRA's rights is more expansive than the law itself. Effectively their trademark policy is to automatically side with the trademark holder, irrespective of fair use[1] (see page 9+).

[0] https://en.wikipedia.org/wiki/Digital_Millennium_Copyright_A...

[1] https://apps.americanbar.org/litigation/committees/intellect...

Re: Highly Available Block Storage

#54

Post by unethical_ban »

cmdrfred wrote: post #39
treehau5 wrote: post #36

Earlier quoted context omitted.

Just a reminder that most businesses would do this exact same thing in this position and the problem is with the policy and enforcement. Get out and vote for the right people this November.

Find me a viable candidate who will work for the people and I will.

Gary Johnson isn't as insane as the other two.

Re: Highly Available Block Storage

#55

Post by mwcampbell »

I think this might be a mistake. Ever since Joyent's commentary on one of the big Amazon EBS failures in 2011 [1] [2] [3], I've been suspicious of all network-attached block storage. Then again, I haven't heard of any big EBS failures recently; I wonder what changed.

[1]: https://www.joyent.com/blog/on-cascading-failures-and-amazon...

[2]: https://www.joyent.com/blog/magical-block-store-when-abstrac...

[3]: https://www.joyent.com/blog/network-storage-in-the-cloud-del...

Re: Highly Available Block Storage

#56

Post by notacoward »

marcstreeter wrote: post #32
scurvy wrote: post #28

What's the backend? Ceph?

Ceph is not block storage -- ceph is object storage

It's fundamentally objects, but not S3-style objects. It's more NASD/T10 objects, with rich semantics including partial writes and strong consistency. Implementing block storage on top of this layer (called RADOS) is trivial. Implementing S3-style objects is a bit more work, and implementing a filesystem was downright hard, but both have been done.

Disclaimer: I'm a Gluster developer, which isn't quite the same as being a Ceph developer but we do talk to one another. ;)

Re: Highly Available Block Storage

#57

Post by scurvy »

marcstreeter wrote: post #32
scurvy wrote: post #28

What's the backend? Ceph?

Ceph is not block storage -- ceph is object storage

Really? Hrmm. Someone should let all those VPS hosters know that they're doing something impossible. Ceph has block, POSIX file, and S3/object interfaces.

Block storage (RBD) is probably the most widely used interface (judging by mailing list and IRC activity).

Since I asked what the "backend" is, my question is still valid.

Re: Highly Available Block Storage

#59

Post by boulos »

3pt14159 wrote: post #43

I have been asking for non-SSD on DO for a long time now. My heart jumped when I saw the HN title, only to be dashed on the rocks. What are us data nerds supposed to do? We want to take 10 terabytes, run a batch process on it, keep the 20TB, then continue with about 5GB of working data until the next month's terabyte comes in, then we want to batch through the 21TB. Right now the price slider doesn't even go up to 21…

Not being snarky, but at that scale why would you still be using Digital Ocean? Are you processing this dataset with just a single droplet?

Are you going to keep each months 10 TiB forever? Something like Nearline is going to be a much better fit even if you process the data and have to pay the $.01/GB retrieval fee.

Disclaimer: I work on compute engine (so of course I want your money), but I'm honestly curious.

Re: Highly Available Block Storage

#60

Post by mikeash »

Mister_Snuggles wrote: post #38

This is EXACTLY the thing that I need for one of my droplets! I love how there is nothing "special" about it - it's just a disk that you can attach to a droplet. I'm sure that under the hood there's some kind of magic going on, but it looks like it's nicely abstracted away. This is what I hoped block storage would turn out to be - here's a block device, use it like one. As soon as this rolls out to the region I've go…

Same here. My $5/month droplet is sufficient for all my needs, except it's a bit limiting on storage. $2/month for an extra 20GB, doubling my storage, would be great. I don't want to migrate anything and I don't care about more CPU or RAM, I just want more space!
Post reply on HN