Earlier quoted context omitted.
It is indeed work that you and your team should be proud of. Any thoughts on rkt?
rkt was needed to push a number of ideas forward in the ecosystem at the time (4 years ago, 2014) and part of its legacy is the creation of technologies that provided plugin interfaces for the container ecosystem. The Container Networking Interface was directly created by the work in rkt and continues on today inside of Kubernetes and the CNCF. This work made it possible for an ecosystem of networking solutions to ex…
Red Hat contributes etcd to the Cloud Native Computing Foundation
11–20 of 38 posts
Re: Red Hat contributes etcd to the Cloud Native Computing Foundation
#12I wonder if this the core of Redhat knowing that the clock is ticking, and make sure that critical software that they worked on is available in an open fashion. One only has to look at Sun and MySQL to see what can happen to a once vibrant open source offering after acquisition.
Re: Red Hat contributes etcd to the Cloud Native Computing Foundation
#13I wonder if this the core of Redhat knowing that the clock is ticking, and make sure that critical software that they worked on is available in an open fashion. One only has to look at Sun and MySQL to see what can happen to a once vibrant open source offering after acquisition.
Re: Red Hat contributes etcd to the Cloud Native Computing Foundation
#14I find that slightly revisionist though I suppose it depends on your definition of "early on".
To my recollection etcd had a very rough patch early on until they overhauled their raft sub-system. Hashicorp caught some flak at the time for giving the raft implementation a hard pass and writing their own. Go ecosystem fragmentation was particularly bad and a very hot topic at the time. I believe that etcd's new found stability and subsequent track record after the new implementation vindicated Hashicorp somewhat.
Re: Red Hat contributes etcd to the Cloud Native Computing Foundation
#15etcd has succeeded as a piece of distributed systems infrastructure beyond our wildest expectation. When Alex Polvi, Xiang Li, and I started the project as a README in the summer of 2013 we identified that their still was no consensus database that was developer friendly, easily secured, production ready, and based on a well understood consensus algorithm. And largely we got lucky with good market timing, the inventi…
Fast-forward 4 months, the project had grown and scaled to support the influx of new curious devs and use-cases that stretched the bounds of what was possible at the time. At the end of the 4-months, we had a 128 node cluster that stayed up for years and still powers all of the emergency notifications in a few states in the US!
Re: Red Hat contributes etcd to the Cloud Native Computing Foundation
#16etcd has succeeded as a piece of distributed systems infrastructure beyond our wildest expectation. When Alex Polvi, Xiang Li, and I started the project as a README in the summer of 2013 we identified that their still was no consensus database that was developer friendly, easily secured, production ready, and based on a well understood consensus algorithm. And largely we got lucky with good market timing, the inventi…
It was a lot of fun watching the project and community evolve. I think you and the team did an excellent job. I remember a huge spike in users around when discovery.etcd.io launched. It was really a game changer for us building large-scale multi-data center telecom systems. I still remember bootstrapping the first cluster in a 24 data center test and having things blow up, particularly in higher-latency environments…
https://github.com/etcd-io/etcd/blob/master/Documentation/pr...
Re: Red Hat contributes etcd to the Cloud Native Computing Foundation
#17Re: Red Hat contributes etcd to the Cloud Native Computing Foundation
#18Earlier quoted context omitted.
Not saying it will happen (just that it usually does), but I almost never see acquired companies immediately turn into the parent. Usually the acquired maintains its course until attrition and cross-pollination replace its original culture with that of the parent. The acquired company eventually exists only as a collection of intellectual property and history. Could take many years depending on how tightly IBM squeez…
I wonder if "We should get this taken care of before IBM takes over" was part of the thought process here.
Uhm, yeah.
Re: Red Hat contributes etcd to the Cloud Native Computing Foundation
#19This is exactly the kind of thing that should be a layer on top of FoundationDB. Too many state stores all doing largely the same thing.
Re: Red Hat contributes etcd to the Cloud Native Computing Foundation
#20> it demonstrated its quality early on I find that slightly revisionist though I suppose it depends on your definition of "early on". To my recollection etcd had a very rough patch early on until they overhauled their raft sub-system. Hashicorp caught some flak at the time for giving the raft implementation a hard pass and writing their own. Go ecosystem fragmentation was particularly bad and a very hot topic at the…
The original go-raft is one of the first raft implementations. At that time, the raft paper was not even officially published. Many other attempts around that time were not very successful, including go-raft.
Making a production ready consensus algorithm is not easy: https://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/.... Things like pipelining, batching, flow control, asynchronous snapshot were not extensively explored in the context of raft. And not much effort has been put into testing due to the immaturity of the applications of raft at the time.
We realized the problem a few months after etcd alpha was initially released and became popular. However, I went back to CMU to continue my master degree for 1yr, which slowed down the progress.
After I came back from school, together with Blake, Yicheng from CoreOS and later on Ben from Cockroach Labs, we built a solid raft impl as our first priority. Once we put etcd/raft inside etcd2, the stability of etcd greatly improved. That is about 1.5 yr after the initial release. Now etcd/raft powers many production level distributed systems: tikv, cockroachdb, dgraph and many others.
Over the last couples of years, the focus of etcd/raft is always stability and nothing else (although people are blaming us for usability :P).