21–30 of 187 posts
I'd be happy to work on a specification with the community.
Earlier quoted context omitted.
I kind of see this as a general trend where CoreOS is concerned they reimplement everything in house, even when open source solution might already exist - rkt, etcd, now this.
what is the existing alternative for storage in distributed container environments?
Mounting containers to a distributed block system is an anti-pattern. This is going to go poorly.
Ceph has had some really smart people working on distributed block for a lot of years, and they still have significant issues. It's not because they're dumb, it's because performant, scalable, and available distributed block is either hard or impossible.
Is the goal to provide POSIX file access as NFS and AFS do? The mention of object storage as a future direction feels like the early "the sky's the limit" euphoria of a new project that has yet to adopt specific end goals.
The initial goals that are in this release:
1. An abstraction for replicated storage between machines in a cluster.
2. A block device "application" on top that an ext4 filesystem could be run on. This is like an EBS.
In the future someone might build other applications like "object storage" or filesystems and we would love to get the feedback on the API to do that. But, that isn't in the initial goals and we will be focusing on the storage and block layers for the time being.
Is the goal to provide POSIX file access as NFS and AFS do? The mention of object storage as a future direction feels like the early "the sky's the limit" euphoria of a new project that has yet to adopt specific end goals.
The project doesn't provide a POSIX filesystem directly. But, instead provides a block device (think AWS EBS). The initial goals that are in this release: 1. An abstraction for replicated storage between machines in a cluster. 2. A block device "application" on top that an ext4 filesystem could be run on. This is like an EBS. In the future someone might build other applications like "object storage" or filesystems an…
Where's the formal specification of the system? If we want these distributed systems to be reliable how can we be sure our algorithms and processes work if we do not first model them? I'd be happy to work on a specification with the community.
This is one of those classic "release too early or release too late" sort of things that got cut in preference to get early feedback and community participation.
Earlier quoted context omitted.
The project doesn't provide a POSIX filesystem directly. But, instead provides a block device (think AWS EBS). The initial goals that are in this release: 1. An abstraction for replicated storage between machines in a cluster. 2. A block device "application" on top that an ext4 filesystem could be run on. This is like an EBS. In the future someone might build other applications like "object storage" or filesystems an…
Will each disk in the system be weighted so that different size disks can be used in the system over time? OpenStack Swift offers this and is useful to not lock-in disk size at design time.
Earlier quoted context omitted.
I kind of see this as a general trend where CoreOS is concerned they reimplement everything in house, even when open source solution might already exist - rkt, etcd, now this.
what is the existing alternative for storage in distributed container environments?
https://en.wikipedia.org/wiki/Clustered_file_system#Distribu...
I don't understand why this is being used over say librados + librbd from Ceph. This seems like an awful lot of work to get the same functionality that Gluster/Ceph already have. Is there something I'm missing here?
A few major things we wanted to accomplish: Backed by etcd. Today etcd is a well tested and widely used consistent store with users like Kubernetes, Flannel (now Canal), Fleet, SkyDNS and many others. Building something like etcd requires tons of testing and etcd is becoming the solid go to for this category of distributed problems. Easy to work on code base. Building an OSS community around a complex technology is r…
To me at least, having it "backed by etcd" is a big red flag, not a feature.
Some things that need to be solved sooner or later: data replication so that N faults of X entities are protected against (X can be disks, enclosures, racks, data centers, regions, ..), recovery from failed disks, scrubbing, data management, backups, some kind of storage orchestration and centralized management.
If you look at Ceph which IMHO represents the state of the art in software defined storage, it took many years to get it to a point that it was usable. I hate to be cynical but in this case I would be surprised if CoreOS can pull this of. I wish them all the best though and would be happy for them if I am wrong.