Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZooKeeper vs. Doozer vs. Etcd

devo.ps

Post reply on HN
Print view View original

61–70 of 91 posts

Re: ZooKeeper vs. Doozer vs. Etcd

#61

Post by voidmain »

philips wrote: post #48
voidmain wrote: post #24

Earlier quoted context omitted.

I agree. At FoundationDB, we're writing a coordination tool (working name "beastmaster") that provides service discovery, locking, leader election, etc on top of our transactional key/value store. It is higher level than these tools; our idea is to try to make it useful from a command line or DNS rather than have to be baked into every piece of software that needs service discovery. beast service --lock name=mailserv…

That is cool! How are you doing locking for transactions right now internally? You can use etcd from the command line with etcdctl or grab environment variables for your process using etcdenv. I would love to see someone back a DNS server with etcd too. etcdctl: https://github.com/coreos/etcdctl etcdenv: https://github.com/mattn/etcdenv

FoundationDB (https://foundationdb.com) provides optimistic transactions-- the client does reads from a consistent snapshot, and then tells the database what it wants to write and what serializable reads it did, and the database figures out whether the transaction can safely commit or whether it needs to be retried.

Beastmaster is just a client that sits on top of FoundationDB's transactions and fault tolerance, and adds coordination-specific things like a global clock, fair locks with timeouts, and a data model for service discovery. And then provides useful command line tools and a simple DNS and REST server. We should have it on github soon.

Re: ZooKeeper vs. Doozer vs. Etcd

#63

Post by vidarh »

Juha wrote: post #35
benjaminwootton wrote: post #11

Earlier quoted context omitted.

Spotify store some configuration in configuration to good effect: http://labs.spotify.com/tag/dns/ It's obviously no Zookeeper but it is proven and mature.

That's interesting use of dns indeed. They also mention the DNS for service discovery approach starts to reach it's limits and Spotify is considering Zookeeper (quote): We have not yet (as of January 2013) started implementing a replacement. We are looking into using Zookeeper as an authoritative source for a static and dynamic service registry, likely with a DNS facade.

I find their reasons curious. Why in the world are they using zone files? There are tons of DNS servers that support database backends, and writing tools to interact with them is easy.

For that matter, writing an authoritative only DNS backend is easy (been there, done that - took about one week from starting to read the RFC's until having a production ready backend; it takes little time because most/all of the hard work is in the recursive resolvers, and the DNS protocol is actively very well described in the RFCs)

And claiming DNS provides a static view of the world is a bit funny - DNS provides TTL values for everything. If you want a dynamic view, you specify low TTLs, and make sure your clients honour them, and couple that with fast replication of the zone data. There's plenty of options for that, from the duct tape (my DNS server could update however many records you could write to disk on your hardware per second - via a small script that used Qmail as a queueing messaging server...) to well polished products.

Couple that with NOTIFY and IXFR, the protocol provides every mechanism necessary for keeping zone data replicated and up to date. Many modern DNS servers also let you instead simply rely on database replication (e.g. you can have the DNS server serve data out of Postgres for example, and use Postgres replication to keep the zones up to date over multiple servers), or leave it to you to do updates.

The appeal with DNS here is the long track record and existence of servers that have been battered to death in far more hostile environments than most internal service discovery systems ever will need to deal with.

The downside to DNS is that to get things like guaranteed consistency, you'd need a backend that can guarantee it, and clients that don't cache (which means you need to be careful about what resolvers you rely on). And then it might be just as easy to just deploy one of the options in this article (but there's nothing inherent with DNS that prevents that either).

Re: ZooKeeper vs. Doozer vs. Etcd

#64

Post by yummyfajitas »

kbd wrote: post #50

Sorry if this is a dumb question but I've never understood the purpose of these configuration stores. If you're not running in the cloud, but have servers in a datacenter that all mount an NFS share, is there any benefit of these over simply reading a json/yaml file off of an NFS mount?

Or if you are running in the cloud, simply reading a json/yaml file off S3?

Re: ZooKeeper vs. Doozer vs. Etcd

#65

Post by jeremyjh »

peterwwillis wrote: post #55

DIY is the pink elephant in the room. Every company in the world that isn't specifically a software-oriented tech company uses some form of DIY model. Actually, strike that, even they use a DIY model. These tools are the proof! Look at the origins for every modern open source management framework or tool, and it was just a DIY tool that some startup-turned-huge-company developed out of their own needs, then cleaned u…

I think its more like a white elephant. I totally understand the motiviation to write and blog about these vanity infrastructure software projects but I agree it is hard to stay enthused about reading about another one in the language du jour.

Re: ZooKeeper vs. Doozer vs. Etcd

#66

Post by cconger »

yummyfajitas wrote: post #64
kbd wrote: post #50

Sorry if this is a dumb question but I've never understood the purpose of these configuration stores. If you're not running in the cloud, but have servers in a datacenter that all mount an NFS share, is there any benefit of these over simply reading a json/yaml file off of an NFS mount?

Or if you are running in the cloud, simply reading a json/yaml file off S3?

I think there are a few key benefits to a centralized manager over the distributed file listener.

1. coordination between services on reconf. 2. Consistent implementation on what constitutes a change through the API on the central server.

By making it API based you can hook into these updates and cause reconfs and coordinated responses (like rolling bounces) through that system as opposed to each system polling the file and hoping that the order comes out in the wash.

You could with enough work make it so that a client was aware of how to handle individual diffs from the file and coordinates through the file but at that point your now distributing common parsing logic across multiple systems (and potentially implementations) rather than a central system which sounds awfully un-DRY.

Re: ZooKeeper vs. Doozer vs. Etcd

#67

Post by regularfry »

kbd wrote: post #50

Sorry if this is a dumb question but I've never understood the purpose of these configuration stores. If you're not running in the cloud, but have servers in a datacenter that all mount an NFS share, is there any benefit of these over simply reading a json/yaml file off of an NFS mount?

Resiliency to your NFS server falling over is one. Sane atomic updates and locking is another.

Re: ZooKeeper vs. Doozer vs. Etcd

#68

Post by rafekett »

kbd wrote: post #50

Sorry if this is a dumb question but I've never understood the purpose of these configuration stores. If you're not running in the cloud, but have servers in a datacenter that all mount an NFS share, is there any benefit of these over simply reading a json/yaml file off of an NFS mount?

Leader election in addition to what else has been said

Re: ZooKeeper vs. Doozer vs. Etcd

#69

Post by kbd »

regularfry wrote: post #67
kbd wrote: post #50

Sorry if this is a dumb question but I've never understood the purpose of these configuration stores. If you're not running in the cloud, but have servers in a datacenter that all mount an NFS share, is there any benefit of these over simply reading a json/yaml file off of an NFS mount?

Resiliency to your NFS server falling over is one. Sane atomic updates and locking is another.

Atomic updates on an NFS share can be achieved with renaming changed files into place, no?

Re: ZooKeeper vs. Doozer vs. Etcd

#70

Post by nemothekid »

lucian1900 wrote: post #41
alanctgardner2 wrote: post #5

They make some cogent points about Zookeeper, but is javaphobia really a valid concern here? Yes, you have to install a JVM, and Oracle doesn't make that as friendly as it could be. But in my experience ZK doesn't bring along "a ton of dependencies". Likewise, I'm skeptical that the performance of Java v. Go in this case makes a huge difference: you're only spinning the JVM once, at startup. Maybe I'm too technically…

Even if you only look at amount of RAM used, with Zookeeper it'll never be lower than 35M and most likely be much higher than 50M. With etcd, you can get away with significantly less than that. The dependencies and JVM boot time are also annoying if you don't already use Java, though.

Are you guys already using etcd in production? I've been using it for a project lately, and I thought it would have low memory usage but it doesn't.

I'm making about ~80 writes/second with only ~125 keys and all 3 of my etcd nodes sit at 200M+ resident memory.

Post reply on HN