Future of Cloud Storage: An inside perspective from an IaaS vendor
1–10 of 18 posts
Re: Future of Cloud Storage: An inside perspective from an IaaS vendor
#2Really interesting to read about SAN versus local storage and also distributed block storage. The latter sounds v. cool, can't wait to see it commercially available!
Re: Future of Cloud Storage: An inside perspective from an IaaS vendor
#3Yes agreed, distributed block storage is definitely the future. Nice to see Linux pioneering this too.
Re: Future of Cloud Storage: An inside perspective from an IaaS vendor
#4Projects like Sheepdog should definitely kill off SANs and RAID. It can't come soon enough, as the post says, current performance in the cloud for storage is so variable. I moved away from EC2 for exactly this reason.
Re: Future of Cloud Storage: An inside perspective from an IaaS vendor
#5Yes agreed, distributed block storage is definitely the future. Nice to see Linux pioneering this too.
Without intending disrespect toward Linux, DRDB-like storage has been around for a number of years.
There are examples that have been in production for at least a dozen years in commercial operating system deployments.
The following is multi-host RAID-1 storage, known as host-based volume shadowing (HBVS) storage:
http://h71000.www7.hp.com/doc/84FINAL/ba554_90020/ba554_9002...
Getting DRDB working is certainly interesting and not a small project, though dealing with the many and various error cases is the truly entertaining part of the effort. Different devices and different failures can and eventually will toss back errors, and with different timing. And from experience with HBVS, these errors can and do shake out in production, and that's bad.
Re: Future of Cloud Storage: An inside perspective from an IaaS vendor
#6Yes agreed, distributed block storage is definitely the future. Nice to see Linux pioneering this too.
Without intending disrespect toward Linux, DRDB-like storage has been around for a number of years. There are examples that have been in production for at least a dozen years in commercial operating system deployments. The following is multi-host RAID-1 storage, known as host-based volume shadowing (HBVS) storage: http://h71000.www7.hp.com/doc/84FINAL/ba554_90020/ba554_9002... Getting DRDB working is certainly intere…
Agreed. I think the post misses out the new file systems supporting the new Linux roll-out. I believe ButterFS is extremely interesting with regards to distributed block storage for example.
Re: Future of Cloud Storage: An inside perspective from an IaaS vendor
#7Earlier quoted context omitted.
Without intending disrespect toward Linux, DRDB-like storage has been around for a number of years. There are examples that have been in production for at least a dozen years in commercial operating system deployments. The following is multi-host RAID-1 storage, known as host-based volume shadowing (HBVS) storage: http://h71000.www7.hp.com/doc/84FINAL/ba554_90020/ba554_9002... Getting DRDB working is certainly intere…
Agreed. I think the post misses out the new file systems supporting the new Linux roll-out. I believe ButterFS is extremely interesting with regards to distributed block storage for example.
By "extremely interesting" do you mean "not distributed at all"?
Re: Future of Cloud Storage: An inside perspective from an IaaS vendor
#8I was wondering about what you guys think of NAS systems.
While I don't claim to fully understand the advantages brought by block storage vs. file based storage, NAS storage should be usable for most applications. If this is true than the clustered or scale out solutions offered by IBM (SONAS), HP (IBRIX) and EMC (with the recent acquisition of Isilon) might be a suitable off the shelf solution matching your scenario (probably there are some open source solutions also that I am not aware of).
The way I understand these solutions you can basically scale capacity and performance independently which in turn translates itself in the flexibility of achieving the ideal balance between capacity and failure point distribution/redundancy.
What do you guys think of this setup? Would it achieve your goals?
I'm coming from a corporate environment where having an off the shelf solution that is supported by some major company is as important as the solution itself.
Re: Future of Cloud Storage: An inside perspective from an IaaS vendor
#9Really interesting to read about SAN versus local storage and also distributed block storage. The latter sounds v. cool, can't wait to see it commercially available!
people keep telling me to use ceph... http://ceph.newdream.net/ but it's obviously not ready for prime time. Like the article said, local storage is still the best price/performance/reliability balance. (I use raid 0+1 rather than raid6)
When distributed storage systems like ceph have been around and in production for 5-10 years, I'll re-evaluate. but for now, they present too much risk of data loss in terms of bugs and admin error.
there's also gluster, which is much closer to my own standards in terms of 'time in production' but even so... local disk is simple, and when it fails it fails in a non-spectacular way.
Of course, the other problem with distributed filesystems is that it makes having a good network /much/ more important. Hell, right now I could get away with 100Mbps, so on a gigabit network, I can have some pretty serious network issues before anyone notices anything is wrong. For a widely distributed storage network? even at my current scale, I'd at least need 10G interconnects between the switches, and god help me if there was a network glitch.
Re: Future of Cloud Storage: An inside perspective from an IaaS vendor
#10I was wondering about what you guys think of NAS systems. While I don't claim to fully understand the advantages brought by block storage vs. file based storage, NAS storage should be usable for most applications. If this is true than the clustered or scale out solutions offered by IBM (SONAS), HP (IBRIX) and EMC (with the recent acquisition of Isilon) might be a suitable off the shelf solution matching your scenario…
The difference between NFS and iSCSI should be fairly small at this point. Anything by IBM, HP, or EMC will cost at least 10x as much as local disk, which is death for most cloud providers. Unfortunately, stuff like DRBD or Sheepdog may be only 2x the cost of local disk but doesn't look ready for prime time.