Another reason to use zetcd over Zookeeper is security. I don't believe it is possible yet to use TLS for Zookeeper-to-Zookeeper communication (clients can connnect to Zookeeper using TLS). The Jira covering this feature [0]. This will increasingly be a problem for those running Zookeeper for Mesos, Kafka etc. from a security and risk point of view.
Introducing zetcd
31–36 of 36 posts
Re: Introducing zetcd
#32Earlier quoted context omitted.
The funny thing about the situation you describe is that there are real world examples of similar circular dependencies. I recall GitHub having an issue like that where their build pipeline used Bower which is hosted on GitHub. When shit hit the fan and a build broke the site, they could build the "fix" as Bower didn't work.
My own experience working at CoreOS is that many of our projects exploit self-referentiality as it's a particularly useful property. Off the top of my head: - Quay.io, our registry service, is built and deployed by itself - Clair, our static analysis tool for detecting security vulnerabilities, analyzes itself - Tectonic, our enterprise Kubernetes distro, is "self-driving" and manages itself - discovery.etcd.io, a se…
I think you are missing the point. It's like running docker registry on kubernetes.
If for some reason the cluster goes down, bootstrapping it might be a bit difficult.
Re: Introducing zetcd
#33A big use case of this that we are thinking about is enabling people to use the etcd Operator[1], which makes it simple to run etcd clusters on Kubernetes, to back their ZooKeeper applications. The neat thing about the etcd Operator is you can define a cluster and the etcd Operator takes care of normal operations by using the Kubernetes API. apiVersion: "etcd.coreos.com/v1beta1" kind: "Cluster" metadata: name: "examp…
So I can run a Kubernetes cluster on Mesos and have the Zookeeper for Mesos deployed on the Kubernetes cluster using the etcd operator and zetcd Joking, of course.
Of course you are joking, but you should keep in mind, that _in the real world_ you would operate the etcd kubernetes needs also via etcd operator!
Re: Introducing zetcd
#34Earlier quoted context omitted.
Who are you targeting with this? The "hesitant" zookeeper folks that already depend on etcd? Are you hoping to unseat cdh here? Pardon the naive question here - I never bump in to k8s selling to traditional enterprise hadoop customers. I'd also never pick kubernetes for my "from scratch" cluster due to already being reliant on the JVM stack. I actually like the idea of giving an IT department that already understands…
Kubernetes is actually getting a solid amount of large-tech and early adopter enterprise deployment. That's still pretty nascent, but it's picking up quickly. Happy to discuss in more details offline the adoption we're seeing. The reason you don't bump into k8s while selling to Hadoop users is that Hadoop isn't something you'd run on a container-based stack (at least not right now and IMO it wont be). There are lots…
>"Pachyderm's whole pitch is that containerized data infra can be really powerful and that enterprises will want to unify their stack to all be containerized and k8s is THE answer for the orchestration layer."
Doesn't Pachyderm predate K8s though? Is this a recent development? Have they shifted focus then?
Re: Introducing zetcd
#35Earlier quoted context omitted.
My own experience working at CoreOS is that many of our projects exploit self-referentiality as it's a particularly useful property. Off the top of my head: - Quay.io, our registry service, is built and deployed by itself - Clair, our static analysis tool for detecting security vulnerabilities, analyzes itself - Tectonic, our enterprise Kubernetes distro, is "self-driving" and manages itself - discovery.etcd.io, a se…
I think you are missing the point. It's like running docker registry on kubernetes. If for some reason the cluster goes down, bootstrapping it might be a bit difficult.
Yes that's the exact point I was trying to make. Things are fine until they're not, at which point it's surgery and tribal knowledge to fix them.