31–40 of 187 posts
Earlier quoted context omitted.
Competition is good, even in open-source land. By the way, I'm still hoping for an open-source competitor for elasticsearch.
> I'm still hoping for an open-source competitor for elasticsearch I thought Elasticsearch was open-sourced, and the GitHub repo says Apache v2. What am I missing here?
I don't understand why this is being used over say librados + librbd from Ceph. This seems like an awful lot of work to get the same functionality that Gluster/Ceph already have. Is there something I'm missing here?
I kind of see this as a general trend where CoreOS is concerned they reimplement everything in house, even when open source solution might already exist - rkt, etcd, now this.
And it generally seems that, when they say "composable modular tools", one should read "includes dependencies on the rest of our stack". This isn't intended as harshness. It is just that every time a tool from COS seems interesting enough to look in to, it turns out that I wouldn't be investing in a tool, it would be investing in an ecosystem that duplicates a ton of what I'm already managing.
I love CoreOS and they've done some super impressive engineering. But really, a new storage system? Rewriting in Go and using etcd for central state management makes things easier, but this is still a hard problem. Some things that need to be solved sooner or later: data replication so that N faults of X entities are protected against (X can be disks, enclosures, racks, data centers, regions, ..), recovery from faile…
Which is why I'm rather surprised that these systems continue to be released without any formal specification or model checking. It's hard to get these things right... worth the effort, IMO, to create formal specifications.
I love CoreOS and they've done some super impressive engineering. But really, a new storage system? Rewriting in Go and using etcd for central state management makes things easier, but this is still a hard problem. Some things that need to be solved sooner or later: data replication so that N faults of X entities are protected against (X can be disks, enclosures, racks, data centers, regions, ..), recovery from faile…
Persistent storage is a "hard problem" Really?
I don't understand why this is being used over say librados + librbd from Ceph. This seems like an awful lot of work to get the same functionality that Gluster/Ceph already have. Is there something I'm missing here?
I kind of see this as a general trend where CoreOS is concerned they reimplement everything in house, even when open source solution might already exist - rkt, etcd, now this.
For Etcd I'm of two minds. I like its simplicity. On the other hand, I have had far more problems with Etcd than I've ever had with e.g. Consul (though I dislike Consuls "and the kitchen sink" approach), up to and including data loss and more than once having to re-initialise clusters. I'm a couple of years of no further problems at least a way from trusting Etcd with my data.
For this? They have to have very compelling stories for why they couldn't e.g. modify Ceph or Gluster to achieve whatever it is that they want. And it's no clear what they want. Gluster for example is built as a stack of pluggable translators that is relatively easy to extend.
Even if they have a credible and compelling story for why we need something else, it will take years of production use by others before I'd trust any data to a new distributed file system.
Gluster and Ceph have those years. I've had a Gluster volume running without maintenance or data loss for more than five years. Even then it took several years before I truly started to trust it.
I love CoreOS and they've done some super impressive engineering. But really, a new storage system? Rewriting in Go and using etcd for central state management makes things easier, but this is still a hard problem. Some things that need to be solved sooner or later: data replication so that N faults of X entities are protected against (X can be disks, enclosures, racks, data centers, regions, ..), recovery from faile…
It's like any time we take a step forward in one area we have to reinvent the last 50 years of computing to support it. Persistent storage is a "hard problem" Really?
Am I understanding correctly that this is a file-based API? Distributing a POSIX filesystem effectively is very challenging, particularly since most applications that use them aren't written with CAP in mind; they don't expect a lot of basic operations to block for extended periods and fail in surprising ways, and they very often perform poorly when operations that are locally quick end up much slower over a network.
To be concrete:
> Today’s Torus release includes manifests using this feature to demonstrate running the PostgreSQL database server atop Kubernetes flex volumes, backed by Torus storage.
It will be interesting to see how well this performs and how it behaves in the face of single-node failures and network congestion.
Storing container images is a great use case for an object store, not a block store. They should get sucked down whole to ephemeral storage on the compute nodes. Mounting containers to a distributed block system is an anti-pattern. This is going to go poorly. Ceph has had some really smart people working on distributed block for a lot of years, and they still have significant issues. It's not because they're dumb, it…
Earlier quoted context omitted.
I kind of see this as a general trend where CoreOS is concerned they reimplement everything in house, even when open source solution might already exist - rkt, etcd, now this.
Rkt had good reasoning behind it, and I think it's reasonable to assume that a lot of the recent changes to Docker came because they forced Docker's hand. For Etcd I'm of two minds. I like its simplicity. On the other hand, I have had far more problems with Etcd than I've ever had with e.g. Consul (though I dislike Consuls "and the kitchen sink" approach), up to and including data loss and more than once having to re…
Earlier quoted context omitted.
what is the existing alternative for storage in distributed container environments?
Ceph, OpenStack Swift, GlusterFS, OrangeFS, Lustre. https://en.wikipedia.org/wiki/Clustered_file_system#Distribu... https://en.wikipedia.org/wiki/Object_storage
I write this not as storage religion (of which there's far too much), but to warn away those who haven't experienced the many kinds of data (and stomach lining) loss that come with being a Lustre admin.