11–20 of 91 posts
DNS on its own can be enough in some scenarios
Care to explain how DNS alone can do distributed config management?
http://labs.spotify.com/tag/dns/
It's obviously no Zookeeper but it is proven and mature.
Earlier quoted context omitted.
Care to explain how DNS alone can do distributed config management?
Oh! Treat records as key-value pairs, and encode the value into the IP. If your timeout is 10 seconds, add a DNS record timeout.server.yourdomain which resolves to an IPv6 address with value 10. It gets tougher with ASCII strings, but you could support multi-record configs as well. Then your application just uses nslookup to download the config when you reload it. If someone builds this I will be their best friend
1. The protocol is difficult to implement. In theory you could just use Jute to codegen this part, but that assumes Jute supports the language you need. Doozer improves on this with a simple text-based protocol, and etcd goes a little further with a HTTP API.
2. The primitives Zookeeper exposes are very primitive. Implementing higher level abstractions such as locking or leader election on top of znodes is easy to get wrong. Just this year Curator has fixed critical bugs in both of those algorithms: https://github.com/Netflix/curator/blob/master/CHANGES.txt
My feeling is that etcd and doozer fare a little better on #2 just because their primitives are slightly easier to understand, but fundamentally the problem still exists. I'm looking forward to seeing more innovation in this area.
They make some cogent points about Zookeeper, but is javaphobia really a valid concern here? Yes, you have to install a JVM, and Oracle doesn't make that as friendly as it could be. But in my experience ZK doesn't bring along "a ton of dependencies". Likewise, I'm skeptical that the performance of Java v. Go in this case makes a huge difference: you're only spinning the JVM once, at startup. Maybe I'm too technically…
As we said, we don't hate Java. We were however constrained in what we could install on the host machines since we administrate these on behalf of others. Even lightweight dependencies were a no-go for us. With regards to Apache, we have "mixed feelings". We're aligned with a lot of what they stand for and are definitely very thankful for what they enabled in the OSS community, but are not enthused with the way they…
They make some cogent points about Zookeeper, but is javaphobia really a valid concern here? Yes, you have to install a JVM, and Oracle doesn't make that as friendly as it could be. But in my experience ZK doesn't bring along "a ton of dependencies". Likewise, I'm skeptical that the performance of Java v. Go in this case makes a huge difference: you're only spinning the JVM once, at startup. Maybe I'm too technically…
Earlier quoted context omitted.
Care to explain how DNS alone can do distributed config management?
Oh! Treat records as key-value pairs, and encode the value into the IP. If your timeout is 10 seconds, add a DNS record timeout.server.yourdomain which resolves to an IPv6 address with value 10. It gets tougher with ASCII strings, but you could support multi-record configs as well. Then your application just uses nslookup to download the config when you reload it. If someone builds this I will be their best friend
Earlier quoted context omitted.
Oh! Treat records as key-value pairs, and encode the value into the IP. If your timeout is 10 seconds, add a DNS record timeout.server.yourdomain which resolves to an IPv6 address with value 10. It gets tougher with ASCII strings, but you could support multi-record configs as well. Then your application just uses nslookup to download the config when you reload it. If someone builds this I will be their best friend
Well, at the very least there's a TXT record for strings. Access control and consistency may both be issues though.
If you've ever wondered why DNS requires the "IN" (for "Internet") in all its record declarations, it's to make this distinction. The other two options are "HE" (for Hesiod), and "CH", for http://en.wikipedia.org/wiki/Chaosnet.
Earlier quoted context omitted.
Care to explain how DNS alone can do distributed config management?
Oh! Treat records as key-value pairs, and encode the value into the IP. If your timeout is 10 seconds, add a DNS record timeout.server.yourdomain which resolves to an IPv6 address with value 10. It gets tougher with ASCII strings, but you could support multi-record configs as well. Then your application just uses nslookup to download the config when you reload it. If someone builds this I will be their best friend
They make some cogent points about Zookeeper, but is javaphobia really a valid concern here? Yes, you have to install a JVM, and Oracle doesn't make that as friendly as it could be. But in my experience ZK doesn't bring along "a ton of dependencies". Likewise, I'm skeptical that the performance of Java v. Go in this case makes a huge difference: you're only spinning the JVM once, at startup. Maybe I'm too technically…
Even though Zookeeper is "mature" it is definitely a beast to fine tune and wade through. Zookeeper definitely does not bring along a ton of dependencies. If you're deploying it as documented it should be running on its own dedicated machines. I don't see why dependencies are a problem there especially with Chef/Puppet nowadays.
Earlier quoted context omitted.
Even though Zookeeper is "mature" it is definitely a beast to fine tune and wade through. Zookeeper definitely does not bring along a ton of dependencies. If you're deploying it as documented it should be running on its own dedicated machines. I don't see why dependencies are a problem there especially with Chef/Puppet nowadays.
Exactly. A "devops" company shouldn't really be complaining about dependencies anyways, they're a fact of life. I admit my experience with ZK has been 50% administering it with Cloudera Manager, which basically abstracts away all the nastiness. I'm curious about specific issues people have had though; I've used ZK with Hadoop and Kafka without any major issues.