Kafka Removing Zookeeper Dependency • Hacker News Board

Re: Kafka Removing Zookeeper Dependency

#151

Post by StreamBright » Sun, May 17, 2020, 6:53 PM UTC

agallego wrote: post #146

StreamBright wrote: post #137

Earlier quoted context omitted.

Which one is easier? Adding a few lines of GC configuration or rewriting a service in C++ using Seastar and binding processes to cores? What sort of performance measurement uses default configurations? What is even the point of not tuning the GC to your application workload? I have spent the last 15 years on running Java apps in production and to optimize for the p99 latency is really not that hard. Optimize p99.9999…

I understand tail latencies - https://www.youtube.com/watch?v=WdFYY3vEcxo - my prev open source project smf ( https://github.com/smfrpc/smf ) uses gill tene's HDR histogram. I think you have a very superficial understanding of seastar. It is not simply marketing, it is a suite of tools and techniques to build low latency software - Specifically for IO intensive apps. Glauber wrote a good into here https://www.scyllad…

I am familiar with Seastar too. It is one component that is pretty useless by itself. What is relevant in this topic is what is around it, the functionality that you provide. This is why Scylla is copying Cassandra. You can come up with a nice way of programming whatever you want but the end of the way the business functionality is what matters and there are different tradeoffs involved, still.

Re: Kafka Removing Zookeeper Dependency

#152

Post by bsaul » Sun, May 17, 2020, 6:59 PM UTC

throwdbaaway wrote: post #142

bsaul wrote: post #98

Earlier quoted context omitted.

And now you have a separate intermediate runtime layer between your code and the OS, with its own set of versions, patches, regression, bugs, backward compatibilities, etc. that may or may not have an impact on the code that you intend to deploy (or redeploy). Statically compiled binaries are undoubtedly a plus regarding deployment.

You do realize that golang statically compiled binaries contain the go runtime right? So, all the concerns that you mentioned are applicable to go as well, e.g. a golang app may be buggy when built with version X, while another golang app may have GC issue when built with version Y, and yet another golang app would be vulnerable unless rebuilt with version Z. [1][2][3] Don't drink the Kool-Aid, golang is just a more…

except that the developer controls it, since its is embedded. As such it follows the exact same lifecycle as the code itself ( versionning, testing, etc).

Re: Kafka Removing Zookeeper Dependency

#153

Post by kitotik » Sun, May 17, 2020, 7:06 PM UTC

tamale wrote: post #135

kitotik wrote: post #120

Earlier quoted context omitted.

> Hardly anything with any amount of complexity is shipped like that The entire HashiCorp stack and Cockroachdb comes to mind. I’m sure there are things that are much more complex in the world, but they do some fairly heavy lifting. > especially not cross-platform I find it drastically simpler to download a single binary for each platform I need to deploy to(macOS,Linux,FreeBSD) than it is to get a consistent Java en…

You're kinda omitting the insane amount of configuration you can give to pretty much every hashicorp product and cockroachdb I think the big difference is that Java projects TEND to be more 'up front' about their configuration options, with shipped default files with a lot of the options already set to some default value, while projects like the ones you mentioned require you to look up every property yourself and se…

I did not mention the configs, because those are plain text and do not really add dependencies I don’t already have.

Are you saying that Java projects tend to have more sane defaults and come bundled with more out of the box?

The last Java project I had to deploy was ElasticSearch/Kibana, and there was a LOT of configuration needed that required consulting a lot of disparate documentation.

Re: Kafka Removing Zookeeper Dependency

#154

Post by Matthias247 » Sun, May 17, 2020, 7:55 PM UTC

agallego wrote: post #62

sz4kerto wrote: post #60

Earlier quoted context omitted.

Because the JVM is fast. And it's not resource hungry - your program might be resource hungry. Bad desktop Java programs misled a whole generation of programmers about the JVM. Look at stuff like LMAX. Java can be lightning fast.

The JVM is fast. (though I'd say the best thing about the JVM is the amazingly large test suite it maintains). I think he was alluding to the fact it is much easier to develop predictable systems in a language that forces you to deal with the constraints up front. - i.e.: writing your own memory allocator for custom pools knowing exactly the latency, throughput, assembly generated for it. Not to mention that a lot of…

> i.e.: writing your own memory allocator for custom pools knowing exactly the latency, throughput, assembly generated for it.

If you want to do this, you can gain a lot of performance with having custom allocators and pools on the JVM as well. E.g. frameworks like Netty have pooling strategies for ByteBuffers. If you go that route, you can also gain a lot of performance on the JVM, might really be competitive.

Unfortunately the JVM still enforces too many heap allocations since value-types are not a thing yet, but it still performs well.

One of interesting things I discovered about C++/Rust vs C#/Java is that the former language family wants you to do a lot of optimizations upfront, wich typically results in good performance. But sometimes you also spend too much time into optimizing something something that won't matter in practice.

Whereas the managed languages are a bit easier to work with by default, but will thereby only yield mediocre performance. However you still have the chance to look into the bottlenecks and improve them by large margins using the right approaches. In C# now even more so than in Java thanks to tools like Span and value types.

Re: Kafka Removing Zookeeper Dependency

#155

Post by Groxx » Sun, May 17, 2020, 8:05 PM UTC

jooz wrote: post #83

weego wrote: post #74

Earlier quoted context omitted.

That's not really a good faith argument. That's the equivalent of saying that a major problem with NodeJS is you have to pick which version to install before you deploy. I deploy single package jvm files most days having not had to consider the deploy environment for a about 4 years.

What is hard is not to choose the java version (1.8,19,...), but the JVM implementation. To choose between hotspot, openjdk, oracle. Even to find the download link without being forced to log in on oracle.com was difficult (in fact I couldnt find it the very last time).

Why is it hard? Since Go gives you no choice, the equivalent would be to pick literally any, and make do with its defaults. Same thing for tuning. You do not have to choose optimally, that's an extra constraint.

Re: Kafka Removing Zookeeper Dependency

#156

Post by sagichmal » Sun, May 17, 2020, 8:06 PM UTC

saurik wrote: post #12

sagichmal wrote: post #8

Earlier quoted context omitted.

Counterpoint, if I need to deploy technology T to solve problem P, I don't want to have to also deploy a flotilla of support technologies because modularity or whatever. ZooKeeper was always an implementation detail of Kafka, the fact that you had to manage it separately was an abstraction leak that I'm happy to see fixed.

I feel like this just argues that ZooKeeper should be a library that can be embedded in the servers of other projects (even if it is conceptually a separate server in the same memory space listening on its own ports; but like, it is otherwise entirely hidden inside of the other program and is entirely managed and configured by it also).

Sounds good to me, more things like ZooKeeper should be embeddable.

Re: Kafka Removing Zookeeper Dependency

#157

Post by Groxx » Sun, May 17, 2020, 8:10 PM UTC

koffiezet wrote: post #92

StreamBright wrote: post #76

Earlier quoted context omitted.

Did you try to tune GC (as most Kafka production systems would have GC configured)? Which JVM version?

Just the fact that you have to ask all these things proves the point?

I'm going to go with "absolutely not".

The fact that a tool works out of the box and offers you an extensive array of ways to get more out of it is not in any way worse than a tool that just works out of the box. You can use it in exactly the same way with no more mental effort - stick to the defaults.

Re: Kafka Removing Zookeeper Dependency

#158

Post by Groxx » Sun, May 17, 2020, 8:12 PM UTC

pjmlp wrote: post #143

gen220 wrote: post #102

Earlier quoted context omitted.

Go’s mission statement isn’t to be faster than Java or to replace java. Its to (1) compile faster (2) compile into single binaries with no dynamic links (3) natively support concurrency and parallelism with M:N routines:threads. Go will never replace Java for Android because (1) it doesn’t use a VM, so it would need to compile for every arch that android runs on (2) it would require a bug-compatible port of Android.…

Java on Android compiles to native code just like Go. Google is writing Android from scratch, is it called Fuchsia and Go also doesn't get to play there. The few parts that were written in Go are scheduled to be rewritten in C++ or Rust, with Dart being the main userspace language.

I honestly have yet to see any evidence that Fuchsia is anything but an experiment in this respect, like Dart was (which was also going to "replace android").

I entirely admit the possibility, but I see no plan. Hopes and aspirations are not plans.

Re: Kafka Removing Zookeeper Dependency

#159

Post by manigandham » Sun, May 17, 2020, 9:24 PM UTC

StreamBright wrote: post #145

manigandham wrote: post #138

Earlier quoted context omitted.

Easier for who? At some point, the architecture needs to change to overcome a fundamental limit. Developers take on that work to make the performance and operations easier for their users. Seastar is the foundation of Scylla, which shows that rewriting in C++ can deliver magnitudes more performance which is not possible by just tuning Cassandra on the JVM. In fact, Datastax has now copied the Scylla approach in Cassa…

>> Easier for who? For any decent SRE out there. >> magnitudes more performance which is not possible by just tuning Cassandra Magnitudes?? Are you talking about the order of magnitude? You should read the ScyllaDB performance report first. https://www.scylladb.com/product/benchmarks/aws-i3-metal-ben... Avg. 99.9% Latency (ms): 9.9 vs. Avg. 99.9% Latency (ms): 474.4 While there is no significant latency difference in…

Yes, you get 10x the throughput while maintaining far better tail latency with Scylla over Cassandra.

How is 10ms vs 475 not a major improvement? How is 4 nodes vs 40 not a major improvement? If you're an SRE than how is managing 4 servers with far less tuning and maintenance not a major improvement? Also 99.9% percentile still matters. They're testing with 300k ops/sec which means 300/sec are facing extreme latency spikes that can be enough to fail and/or cause cascading issues through out the application.

There's no metric where Cassandra is better here and you can't tune your way to the same performance in the first place which is the whole point of Scylla. What even is your claim here? Spend more to get less?

Re: Kafka Removing Zookeeper Dependency

#160

Post by pjmlp » Sun, May 17, 2020, 9:27 PM UTC

Groxx wrote: post #158

pjmlp wrote: post #143

Earlier quoted context omitted.

Java on Android compiles to native code just like Go. Google is writing Android from scratch, is it called Fuchsia and Go also doesn't get to play there. The few parts that were written in Go are scheduled to be rewritten in C++ or Rust, with Dart being the main userspace language.

I honestly have yet to see any evidence that Fuchsia is anything but an experiment in this respect, like Dart was (which was also going to "replace android"). I entirely admit the possibility , but I see no plan . Hopes and aspirations are not plans.

> That being said, if you were writing Android from scratch and were only targeting ARM (an equally silly comparison meant to highlight the differences in the languages), you’d be hard pressed to justify Java over Go.

Instead they went with Rust, C++ and Dart.

Now what have all those languages in common that Go lacks?