Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

coreos.com

Post reply on HN
Print view View original

41–50 of 50 posts

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#41

Post by nl »

ballard wrote: post #35
vidarh wrote: post #29

The discovery mechanism looks clumsy to me. There's no way we'd rely on a public discovery service, for example. If we're going to hardcode configuration information - such as the URL of a discovery service that may or may not be up or reachable, we might as well hardcode the addresses of a few peers. And running a second etcd cluster to bring up the main one seems pointless. Either it's turtles all the way down, or…

This is why top-down, single-source of truth is mostly the best way for core infrastructure. No pun intended. It's important for some parts to not be infinitely reconfigurable or dynamic because it would create chaos and service dependency deadlock/DoS. Most people not running bare metal don't know the lessons of how and why things underneath are they way they are and the sensible limits what's possible. It's also mo…

This is why top-down, single-source of truth is mostly the best way for core infrastructure.

What happens when your single source of truth goes down?

etcd attempts to provide a source of truth by using the Raft[1] consensus algorithm to determine that "truth" in an environment when relying on merely one place is too risky.

Th Raft and Paxos[2] (as used in Zookeeper & Google Chubby) algorithms give guarantees for consensus in an environment with unreliable hardware. That is better than a single source of truth where you get nothing when that source disappears.

[1] http://raftconsensus.github.io/

[2] http://en.wikipedia.org/wiki/Paxos_(computer_science)

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#42

Post by vidarh »

ballard wrote: post #34
vidarh wrote: post #31

Earlier quoted context omitted.

> It doesn't coordinate the transition of services to other states. It's not intended to. It is way outside the scope. But you can easily coordinate transitions on top of Etcd by waiting on key changes. > Configuration management is as much management order of operations as well as what files should contain what. Outside of scope. Nothing stops you from using a configuration management engine like Puppet and triggeri…

Trying to cast an argument as a strawman with a strawman is pretty weak. You've also admitted that it's an incomplete solution. But What's worse is that people are going to buy this and quickly they'll have to scrap it when it gets out of sync, deadlocks or can't handle more than 8 boxes.

> Trying to cast an argument as a strawman with a strawman is pretty weak.

I've not presented a strawman. I've pointed out to you that people have different needs. I don't need, nor want what Puppet provides, and Etcd does not claim to provide what Puppet provides.

I do, however want what Etcd provides.

> You've also admitted that it's an incomplete solution

In the sense that a car is an incomplete solution if you need a space shuttle.

Yet the car is still more practical if you want to drive on a road.

> But What's worse is that people are going to buy this and quickly they'll have to scrap it when it gets out of sync, deadlocks or can't handle more than 8 boxes.

What basis do you have for that claim?

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#43

Post by vidarh »

ballard wrote: post #35
vidarh wrote: post #29

The discovery mechanism looks clumsy to me. There's no way we'd rely on a public discovery service, for example. If we're going to hardcode configuration information - such as the URL of a discovery service that may or may not be up or reachable, we might as well hardcode the addresses of a few peers. And running a second etcd cluster to bring up the main one seems pointless. Either it's turtles all the way down, or…

This is why top-down, single-source of truth is mostly the best way for core infrastructure. No pun intended. It's important for some parts to not be infinitely reconfigurable or dynamic because it would create chaos and service dependency deadlock/DoS. Most people not running bare metal don't know the lessons of how and why things underneath are they way they are and the sensible limits what's possible. It's also mo…

> This is why top-down, single-source of truth is mostly the best way for core infrastructure.

"Top-down, single-source of truth" needs to deal with failover for resilience in any serious setup. The entire point of consensus algorithms like Raft (and Zab, since you've mentioned ZooKeeper) is that they provide a method of providing a consistent source of truth that is resilient without the management overhead.

It's explicitly because top-down solutions are a massive hassle to keep consistent and available in the face of stalled replication (OpenLDAP.... DNS based solutions...) or a failed master.

> It's important for some parts to not be infinitely reconfigurable or dynamic because it would create chaos and service dependency deadlock/DoS.

That is entirely irrelevant to the algorithm used to replicate the configuration information.

> It's also more secure because if it's possible to dictate all of the service details, it's much easier to run a lean and locked-down infrastructure.

Why exactly do you believe that it is not "possible to dictate all of the service details" while using Etcd? To me it's an entirely orthogonal issue.

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#44

Post by vidarh »

uniclaude wrote: post #39
nl wrote: post #28

Earlier quoted context omitted.

I'm not sure what problem you think etcd is attempting to solve, but if your solution involvs Puppet or Chef then it isn't surprising you find etcd isn't sufficient for your purposes. etcd is a highly available configuration store. That's all. It's designed for applications to use to store configuration, so they can be started & stopped without relying on file based storage. Generally speaking these applications have…

No bashing against etcd from me, but can you point me to a reason why a regular database wouldn't work for this type of thing? High availability, throughput and data storage seem to fit into what a database is made for.

You'd think so, but most RDBMS's are a lot of work to set up in a way that provides the kind of consistency guarantees and availability that a configuration store like this does.

E.g. take Postgres. You can't connect to the slaves and write, so your client will need to know how to identify which server is currently the master. And if the master fails, you need to implement a leadership election method, promote the chosen slave, and resync your other slaves from the new master.

Before you know it you've reimplemented the configuration store with a different data storage mechanism. There's very little intersection between the typical RDBMS and a configuration store like this.

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#45

Post by vidarh »

ballard wrote: post #21
jarito wrote: post #6

Earlier quoted context omitted.

It is a distributed configuration management clustering product like zookeeper. It simplifies the creation and sharing of mostly configuration data in clustered or distributed systems.

It's trying to solve with marketing what Chef does. There is more to configuration management that templating files, but in inspection and having other services arriving at other states.

It is not even trying to solve remotely the same issues as Chef. For starters, it is not going to install or uninstall packages or modify files, or nearly any of the things that a typical Chef recipe might do.

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#46

Post by uniclaude »

vidarh wrote: post #44
uniclaude wrote: post #39

Earlier quoted context omitted.

No bashing against etcd from me, but can you point me to a reason why a regular database wouldn't work for this type of thing? High availability, throughput and data storage seem to fit into what a database is made for.

You'd think so, but most RDBMS's are a lot of work to set up in a way that provides the kind of consistency guarantees and availability that a configuration store like this does. E.g. take Postgres. You can't connect to the slaves and write, so your client will need to know how to identify which server is currently the master. And if the master fails, you need to implement a leadership election method, promote the ch…

Well, even with your point there are a couple reasons why I could imagine databases making sense here.

One is that you might already have people used to deal with them in a highly available manner in your team. HA with Postgres is not exactly an area of research anymore. Most people already using Postgres in their stack know about tools like pgbouncer.

Another one is that there are other databases, like Riak, that can talk http and have those problems solved.

I am here more trying to understand the features that would make one pick etcd over an higly available database. Thanks for taking the time to explain.

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#47

Post by nl »

uniclaude wrote: post #46
vidarh wrote: post #44

Earlier quoted context omitted.

You'd think so, but most RDBMS's are a lot of work to set up in a way that provides the kind of consistency guarantees and availability that a configuration store like this does. E.g. take Postgres. You can't connect to the slaves and write, so your client will need to know how to identify which server is currently the master. And if the master fails, you need to implement a leadership election method, promote the ch…

Well, even with your point there are a couple reasons why I could imagine databases making sense here. One is that you might already have people used to deal with them in a highly available manner in your team. HA with Postgres is not exactly an area of research anymore. Most people already using Postgres in their stack know about tools like pgbouncer. Another one is that there are other databases, like Riak, that ca…

They solve different problems.

In CAP theorem[1] terms, etcd provides Consistency and Partition-Tolerance, while databases generally provide Consistency and Availability.

That's important, because it shows the kind of problem each is trying to solve.

[1] http://en.wikipedia.org/wiki/CAP_theorem

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#48

Post by ithkuil »

ballard wrote: post #18
Touche wrote: post #7

Earlier quoted context omitted.

I don't know what that means or what zookeeper is... Can you explain the problem it solves?

Zookeeper is a very specialized distributed filesystem for saving and delivering important details for very large systems. Getting concurrent distributed systems like this right is the equivalent of summited Mount Everest starting by walking from Paris. It was mostly developed at Yahoo and spun off as an Apache project under Hadoop and commercialized by Cloudera. Netflix uses ZK for example.

so you are basically saying that you don't trust the people who designed etcd.

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#49

Post by uniclaude »

nl wrote: post #47
uniclaude wrote: post #46

Earlier quoted context omitted.

Well, even with your point there are a couple reasons why I could imagine databases making sense here. One is that you might already have people used to deal with them in a highly available manner in your team. HA with Postgres is not exactly an area of research anymore. Most people already using Postgres in their stack know about tools like pgbouncer. Another one is that there are other databases, like Riak, that ca…

They solve different problems. In CAP theorem[1] terms, etcd provides Consistency and Partition-Tolerance, while databases generally provide Consistency and Availability. That's important, because it shows the kind of problem each is trying to solve. [1] http://en.wikipedia.org/wiki/CAP_theorem

Great, that's exactly the answer I was looking for. Now everything makes sense.

Re: Etcd 0.3.0 – Improved Cluster Discovery, API Enhancements and Windows Support

#50

Post by garyrichardson »

vidarh wrote: post #29

The discovery mechanism looks clumsy to me. There's no way we'd rely on a public discovery service, for example. If we're going to hardcode configuration information - such as the URL of a discovery service that may or may not be up or reachable, we might as well hardcode the addresses of a few peers. And running a second etcd cluster to bring up the main one seems pointless. Either it's turtles all the way down, or…

"Turtles all the way down" made my day. Brought me back to my highschool days of reading all of Hawking's books.
Post reply on HN