Etcd – The Road to 1.0
coreos.com
Etcd – The Road to 1.0
1–10 of 27 posts
Re: Etcd – The Road to 1.0
#2Is there any benefit to using coreOS when you don't need a million machines? How much work is it to start with, for example, if you have no idea what your scaling needs will be in the future?
Re: Etcd – The Road to 1.0
#3Is there any benefit to using coreOS when you don't need a million machines? How much work is it to start with, for example, if you have no idea what your scaling needs will be in the future?
No work at all; basic coreos is just docker managed by systemd. Can you make your application docker-izable? Were you planning on using process management? If you answered yes to both of these questions... :)
Further, coreos tries to make you write applications in a 12-factor-y way, so when the time does come for you to use a million machines, you won't need to make huge adjustments to your deployments (just plug the container and init script into fleet and let it roll).
Re: Etcd – The Road to 1.0
#4Is there any benefit to using coreOS when you don't need a million machines? How much work is it to start with, for example, if you have no idea what your scaling needs will be in the future?
The main benefit is it's an easy to update, minimal OS. And docker's already set up for you and updated regularly.
Our cloud-config implementation is designed so that you can use the same config to set up a new machine to match an existing one, and have it automatically join the cluster. Start with 3 machines and scale up as you need.
Learning systemd is one bump you'll have to get over, but all of the major distros will be using it, so they're is no better time to try it out.
Re: Etcd – The Road to 1.0
#5Is there any benefit to using coreOS when you don't need a million machines? How much work is it to start with, for example, if you have no idea what your scaling needs will be in the future?
I'm interested in it because I plan to use docker to ease application management, and with CoreOS I barely have to maintain the operating system because it's so lightweight. So it greatly simplifies things :)
If you think docker is a good fit for your application, this is kind of the next step.
Re: Etcd – The Road to 1.0
#6Is there any benefit to using coreOS when you don't need a million machines? How much work is it to start with, for example, if you have no idea what your scaling needs will be in the future?
It is not a lot of work to get started, here is a short and simple tutorial that goes from nothing to a full running app with a database on CoreOS: http://www.centurylinklabs.com/building-your-first-app-on-co...
Re: Etcd – The Road to 1.0
#7I know that there are lots of heavyweight folks who swear by Zookeeper as both a reliable and powerful tool, and for good reason. Unfortunately the docs can be fairly inscrutable, even for experts, and it typically requires the maintenance of a separate cluster of Zookeeper nodes.
So I like that etcd is a fundamental component of CoreOS, with these features:
1. Written from scratch in Go
2. Implements the Raft protocol for node coordination
3. Has a useful command line app
4. Runs on every node in a cluster
5. Enables auto-discovery
6. Allows CoreOS nodes to share stateRe: Etcd – The Road to 1.0
#8I know that there are lots of heavyweight folks who swear by Zookeeper as both a reliable and powerful tool, and for good reason. Unfortunately the docs can be fairly inscrutable, even for experts, and it typically requires the maintenance of a separate cluster of Zookeeper nodes. So I like that etcd is a fundamental component of CoreOS, with these features: 1. Written from scratch in Go 2. Implements the Raft protoc…
> 4. Runs on every node in a cluster
Is that so? When looking at it I distinctly remember it advising running a set of 3-9 nodes of etcd (not necessarily separate from other things).
Re: Etcd – The Road to 1.0
#9I know that there are lots of heavyweight folks who swear by Zookeeper as both a reliable and powerful tool, and for good reason. Unfortunately the docs can be fairly inscrutable, even for experts, and it typically requires the maintenance of a separate cluster of Zookeeper nodes. So I like that etcd is a fundamental component of CoreOS, with these features: 1. Written from scratch in Go 2. Implements the Raft protoc…
> 4. Runs on every node in a cluster Is that so? When looking at it I distinctly remember it advising running a set of 3-9 nodes of etcd (not necessarily separate from other things).
The functionality to handle this is mentioned in the blog post: "standby" peer mode.
"Our upcoming release, etcd 0.4, adds a new feature, standby mode. It is an important first step in allowing etcd clusters to grow beyond just the peers that participate in consensus."
Re: Etcd – The Road to 1.0
#10I know that there are lots of heavyweight folks who swear by Zookeeper as both a reliable and powerful tool, and for good reason. Unfortunately the docs can be fairly inscrutable, even for experts, and it typically requires the maintenance of a separate cluster of Zookeeper nodes. So I like that etcd is a fundamental component of CoreOS, with these features: 1. Written from scratch in Go 2. Implements the Raft protoc…
> 4. Runs on every node in a cluster Is that so? When looking at it I distinctly remember it advising running a set of 3-9 nodes of etcd (not necessarily separate from other things).
I don't know that that's necessarily a good idea.
As you (perhaps automatically) expand and collapse the cluster, you'll need to make sure to communicate to all nodes what the new cluster size is. If some nodes don't know the correct quorum count, split-brain!
Also, coordination services are typically critical, so its important to isolate from to the bugs in the adhoc code you're writing for your web tier, a crazy query in your database, etc.
It's much easier and safer in practice to just have 3 or 5 nodes running the coordination in isolation.
Edit: more reasons -- It's easier to deploy a coordination service to 5 nodes than 500. It's easier to debug 5 nodes than 500.