Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZooKeeper vs. Doozer vs. Etcd

devo.ps

Post reply on HN
Print view View original

71–80 of 91 posts

Re: ZooKeeper vs. Doozer vs. Etcd

#71

Post by kbd »

rafekett wrote: post #68
kbd wrote: post #50

Sorry if this is a dumb question but I've never understood the purpose of these configuration stores. If you're not running in the cloud, but have servers in a datacenter that all mount an NFS share, is there any benefit of these over simply reading a json/yaml file off of an NFS mount?

Leader election in addition to what else has been said

Leader election - isn't that just a problem created by using an external cluster of servers for storing config? If you were just reading a file, there's no issue of selecting a master from your NFS share.

The two rationales I've heard so far are A. a cluster of one of these config servers will provide better uptime than just an NFS share and B. you can use one of these things as a distributed lock server.

Re: ZooKeeper vs. Doozer vs. Etcd

#72

Post by mattzito »

Xorlev wrote: post #43
nivertech wrote: post #15

Earlier quoted context omitted.

It's not javaphobia. I think the point is that Go produces a statically-linked native executable, while VM-based languages like Java adding many dependencies and also reduce memory which left for the application itself (assuming z-nodes running on the same nodes as application).

You can run a high performance Java process with a small heap, I would call this a non-issue. I have many heavily-used services with 32-64mb of heap space. Certainly, Go will end up using less memory but in the day and age of small servers with 1.5GB of RAM, it's really a non-issue.

Right, but bear in mind that their customers will see that ram usage as "wasted" RAM.

I have a product with an agent that runs around 80MB of RAM usage on giant servers, servers with 500+GB of RAM, and my customers still complain sometimes.

Re: ZooKeeper vs. Doozer vs. Etcd

#74

Post by unoti »

nemothekid wrote: post #70
lucian1900 wrote: post #41

Earlier quoted context omitted.

Even if you only look at amount of RAM used, with Zookeeper it'll never be lower than 35M and most likely be much higher than 50M. With etcd, you can get away with significantly less than that. The dependencies and JVM boot time are also annoying if you don't already use Java, though.

Are you guys already using etcd in production? I've been using it for a project lately, and I thought it would have low memory usage but it doesn't. I'm making about ~80 writes/second with only ~125 keys and all 3 of my etcd nodes sit at 200M+ resident memory.

Is 80 writes per second super huge traffic, beyond the realm of what this is designed to handle? I would expect for etcd to be used for things like which machines are responsible for doing what in a cluster, and for the values to change relatively infrequently during events such as adding a new machine to the cluster. Perhaps for a system running 80 requests per second, using etcd to locate a Redis server would be better. Would the memory use be more reasonable then, if it weren't replicating such a huge volume of log transactions?

Re: ZooKeeper vs. Doozer vs. Etcd

#75

Post by nknighthb »

kbd wrote: post #69
regularfry wrote: post #67

Earlier quoted context omitted.

Resiliency to your NFS server falling over is one. Sane atomic updates and locking is another.

Atomic updates on an NFS share can be achieved with renaming changed files into place, no?

It sounds like you're expecting NFS to be a reliable, proven, high-performance technology. It's anything but. I've never encountered an implementation without horrible bugs, most commonly involving locking and atomicity issues. Things get oh-so-much worse if you have to mix implementations, and performance is always bad.

I would never consider NFS an option in any production system.

On a more general note, anything you build that relies on the particular semantics of any sort of traditional filesystem is sure to be wrong, either now or in the future when it needs to be run on a different filesystem. This is an area of software engineering that's a serious pain in the ass. Avoid it whenever you can.

Re: ZooKeeper vs. Doozer vs. Etcd

#76

Post by nieksand »

babo wrote: post #33
tomjohnson3 wrote: post #21

Earlier quoted context omitted.

one other thing to keep in mind is that the underlying algorithms (zab and raft) provide different guarantees. for example, zookeeper/zab allows reading directly from followers with a guarantee to get at least a past value that won't be rolled back. this was one reason zookeer didn't use paxos: https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zab+vs... in my understanding, raft doesn't allow reading directly fro…

> raft doesn't allow reading directly from followers, In raft all client connections to followers redirected to use the current master. > i think they did paxos a disadvantage by not just focusing on multi-paxos Raft is equivalent to (multi-)Paxos

Having followers redirect to the leader is how the paper describes the algorithm.

But I don't think there is anything stopping you from having followers service committed log entry reads, provided you're willing to live with being out-of-date.

Re: ZooKeeper vs. Doozer vs. Etcd

#77

Post by nemothekid »

unoti wrote: post #74
nemothekid wrote: post #70

Earlier quoted context omitted.

Are you guys already using etcd in production? I've been using it for a project lately, and I thought it would have low memory usage but it doesn't. I'm making about ~80 writes/second with only ~125 keys and all 3 of my etcd nodes sit at 200M+ resident memory.

Is 80 writes per second super huge traffic, beyond the realm of what this is designed to handle? I would expect for etcd to be used for things like which machines are responsible for doing what in a cluster, and for the values to change relatively infrequently during events such as adding a new machine to the cluster. Perhaps for a system running 80 requests per second, using etcd to locate a Redis server would be be…

The readme claims that etcd's can perform up top 1000 w/s, and I don't think 80/s is particularly alot.

At the end of the day there are only ~125 keys (most of which aren't the 80, expire really quickly), and I'm only writing integers (maybe 8 bytes in length converted to strings).

However, I dont know if this is caused my use case or if its just a normal day for etcd. I haven't been able to confirm with anyone else what their memory patterns look like. While running my etcd cluster of the last couple of days my log file has grown 3x to 92M (expected), but my memory usage hasn't grown that much. It now sits at 295M, so I'm unconvinced it has anything to actually to do with the way I'm using it.

I opened an issue here, https://github.com/coreos/etcd/issues/162, and it seems the maintainer confirmed it.

Re: ZooKeeper vs. Doozer vs. Etcd

#78

Post by SEJeff »

philsnow wrote: post #45
russell_h wrote: post #13

One point the article didn't cover is clients. Making a good Zookeeper client is hard, for two reasons: 1. The protocol is difficult to implement. In theory you could just use Jute to codegen this part, but that assumes Jute supports the language you need. Doozer improves on this with a simple text-based protocol, and etcd goes a little further with a HTTP API. 2. The primitives Zookeeper exposes are very primitive.…

re #1, as I said elsewhere, at youtube we use zkocc to proxy readonly connections to zookeeper, and zkocc supports e.g. bson-over-http (and in any case it would be easier to add new client protocol support to zkocc than to zookeeper, I think. maybe I just think that because I don't want to java.)

You work for youtube and use zookeeper?

Re: ZooKeeper vs. Doozer vs. Etcd

#79

Post by codexon »

babo wrote: post #33
tomjohnson3 wrote: post #21

Earlier quoted context omitted.

one other thing to keep in mind is that the underlying algorithms (zab and raft) provide different guarantees. for example, zookeeper/zab allows reading directly from followers with a guarantee to get at least a past value that won't be rolled back. this was one reason zookeer didn't use paxos: https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zab+vs... in my understanding, raft doesn't allow reading directly fro…

> raft doesn't allow reading directly from followers, In raft all client connections to followers redirected to use the current master. > i think they did paxos a disadvantage by not just focusing on multi-paxos Raft is equivalent to (multi-)Paxos

Does this mean that there is no read scaling?

The master server must be able to handle the entire read load?

Re: ZooKeeper vs. Doozer vs. Etcd

#80

Post by SEJeff »

maplebed wrote: post #54

I chose Zookeeper to achieve this same goal a while ago (before I heard of etcd). I have been pleasantly surprised at how useful having a coordination service is in my infrastructure in addition to a configuration management service. Because of this, even though etcd looks like it serves distributed configuration management better (aka simpler), I'm happy with my choice of Zookeeper. Two examples of how a coordinatio…

How do you power cron from ZK? Do you use something like airbnb's chronos[1]?

[1] http://nerds.airbnb.com/introducing-chronos

Post reply on HN