41–50 of 121 posts
I was interested until I saw the Zookeeper dependency. I have had too many deployment nightmares with Zookeeper. I would prefer to avoid it as much as possible, plus systems software in Java, sigh.
I was interested until I saw the Zookeeper dependency. I have had too many deployment nightmares with Zookeeper. I would prefer to avoid it as much as possible, plus systems software in Java, sigh.
I was interested until I saw the Zookeeper dependency. I have had too many deployment nightmares with Zookeeper. I would prefer to avoid it as much as possible, plus systems software in Java, sigh.
This could be an interesting competitor to Apache Kafka, which is singularly unique in this space as far as I'm aware. On another note, I find it somewhat funny that these are called "log" services, logging is probably the least interesting use case for these things I can think of. A better description in my mind would be as a distributed event processing framework, since what they are really doing is distributing di…
I don't think "distributed event processing framework" describes this system at all. It's exactly what it says on the tin: a replicated log (i.e. event log) storage system that one can build other systems on top of. It's not processing events as far as I can see?
https://en.wikipedia.org/wiki/Log-structured_file_system
The use of the term "log" in some contexts implies concepts like lossy, unimportant, non-durable, etc. However, in the context of log-structured file systems or journaling file systems, the sense that's intended is append-only as a primitive, atomic, consistent, durable (replicated) operation.
Earlier quoted context omitted.
Because Scala adds overhead.
Where? In operational cost? In development time? In performance?
Depending on use-case, of course, there's lot of places where scala is preferable to Java. A byte-shuttling service where performance is important, mutability reigns and expressiveness is irrelevant.. not one of those places.
Earlier quoted context omitted.
Which says " At design time we had concerns about Kafka’s I/O model and its lack of strong durability guarantees‐a non-starter for an application like a distributed transaction log[3]" Seems reasonable, right? Except "[3] Kafka addressed these durability concerns in version 0.8" So they built a whole thing, because they didn't bother to ask or say "hey, if we help fix the durability, would that we welcome?" or even "…
Arguably there are benefits to developing in-house expertise, and no better way to develop expertise than to architect and build a solution end-to-end. Twitter now has several domain experts on staff who can continue maintaining DistributedLog and/or weigh its benefits against Kafka's and make more informed decisions going forward. Not saying that they couldn't have worked more closely with Kafka's team in the first…
CantTellIfSerious.jpg
Earlier quoted context omitted.
Seriously this seems like a common pattern in open source: [big company] could just improve [X] but instead builds something from the ground up.
Sometimes you need to let talented engineers build things from the ground up, because it's good for them, makes them happy, and stops them from going to work somewhere else. Keeping people with the skills to solve these types of problems around and happy is also great for recruiting and for helping your less capable engineers learn and grow.
What's the point of retaining an engineer who's doing nothing for the business but re-inventing existing successful software?
I was interested until I saw the Zookeeper dependency. I have had too many deployment nightmares with Zookeeper. I would prefer to avoid it as much as possible, plus systems software in Java, sigh.
I was interested until I saw the Zookeeper dependency. I have had too many deployment nightmares with Zookeeper. I would prefer to avoid it as much as possible, plus systems software in Java, sigh.
In terms of simplifying the deployment, they could have went embedded with Atomix[1] instead. Perhaps next time.
[1]: http://atomix.io/