Introducing zetcd
coreos.com
Introducing zetcd
1–10 of 36 posts
Re: Introducing zetcd
#2Interesting. Maybe hashicorp should release an etcd compatible layer for Consul. Or CockroachDB even. Both have one thing etcd does not...inbuilt WAN support.
Re: Introducing zetcd
#3Great idea, we only run Zookeeper because of Kafka but everything else is in Consul so the idea of this existing is pretty neat.
Re: Introducing zetcd
#4A few years ago I tried to reimplement the server connections directly in Zookeeper to be HTTP connections and a REST-like API and try to use long-polling and some other things. I got some code written, but no prototype up and I had to put it aside. This was initially before etcd was really going (I think it was at version 0.2 when I started the work). I had always hoped somebody else would try the same thing, but no such luck.
Re: Introducing zetcd
#5A big use case of this that we are thinking about is enabling people to use the etcd Operator[1], which makes it simple to run etcd clusters on Kubernetes, to back their ZooKeeper applications.
The neat thing about the etcd Operator is you can define a cluster and the etcd Operator takes care of normal operations by using the Kubernetes API.
apiVersion: "etcd.coreos.com/v1beta1"
kind: "Cluster"
metadata:
name: "example-etcd-cluster"
spec:
size: 5
version: "3.1.8"
Anyways, the zetcd project is still super young but would love more folks to try it out. As the post says folks have already tried using Kafka, Mesos, and others.
[1] https://coreos.com/blog/introducing-the-etcd-operator.html
Re: Introducing zetcd
#6Interesting. Maybe hashicorp should release an etcd compatible layer for Consul. Or CockroachDB even. Both have one thing etcd does not...inbuilt WAN support.
etcd can cross WAN links with tuning for the expected latencies. Tuning latencies is required to ensure the leader election algorithms know when to trigger a failure[1].
What is your use case?
Re: Introducing zetcd
#7A big use case of this that we are thinking about is enabling people to use the etcd Operator[1], which makes it simple to run etcd clusters on Kubernetes, to back their ZooKeeper applications. The neat thing about the etcd Operator is you can define a cluster and the etcd Operator takes care of normal operations by using the Kubernetes API. apiVersion: "etcd.coreos.com/v1beta1" kind: "Cluster" metadata: name: "examp…
Oh, and here is a video explanation of the etcd Operator https://youtu.be/Uf7PiHXqmnw?t=11
Re: Introducing zetcd
#8Interesting. Maybe hashicorp should release an etcd compatible layer for Consul. Or CockroachDB even. Both have one thing etcd does not...inbuilt WAN support.
etcd can cross WAN links with tuning for the expected latencies. Tuning latencies is required to ensure the leader election algorithms know when to trigger a failure[1]. What is your use case? [1] https://coreos.com/etcd/docs/latest/tuning.html
Yes, you can sort of detune the whole cluster. That's not quite the same as Consul's and CockroachDB's specific WAN awareness. Those two take different approaches, but do specifically understand and compensate.
>What is your use case?
Contract work, so use case varies. I agree that etcd is often the right answer.
Re: Introducing zetcd
#9Earlier quoted context omitted.
etcd can cross WAN links with tuning for the expected latencies. Tuning latencies is required to ensure the leader election algorithms know when to trigger a failure[1]. What is your use case? [1] https://coreos.com/etcd/docs/latest/tuning.html
Yes, you can sort of detune the whole cluster. That's not quite the same as Consul's and CockroachDB's specific WAN awareness. Those two take different approaches, but do specifically understand and compensate. >What is your use case? Contract work, so use case varies. I agree that etcd is often the right answer.
Right, etcd's entire focus is on being a consensus database for distributed systems needing coordination, locking, etc. So, eventually consistent WAN replication hasn't really been a focus.
I do think this sort of cross-cluster key replication is useful and we offer it as a userspace external tool called make-mirror[1].
[1] https://github.com/coreos/etcd/blob/master/etcdctl/README.md...
Re: Introducing zetcd
#10So how does zetcd handle the zookeeper session? It's in zetcd and then the "ephemeral nodes" are removed from etcd if the zk client session expires? This is handled by the proxy?
This seems to be the major design compatibility item between the two is that the zookeeper protocol has a session state and so supports ephemeral nodes. If a gc pauses the jvm for a long time the client session will expire. In etcd there is ttl instead (zookeeper always lacked ttl as it wasn't needed)