Kafka Removing Zookeeper Dependency • Hacker News Board

Re: Kafka Removing Zookeeper Dependency

#81

Post by michaelt » Sun, May 17, 2020, 8:18 AM UTC

tiew9Vii wrote: post #73

silisili wrote: post #59

Earlier quoted context omitted.

I don't understand Java apologists. I'm not saying Java is bad. Quite the opposite, it's one of the most optimized systems to date. But if you cannot admit Java apps are a pain to deploy, or, at least harder than deploying a static binary, I cannot take this argument seriously in good faith.

They are really simple if done correctly. You have a single fat jar so can do java -jar myapp.jar which is almost as easy as your static binary but prefixed with two extra words. It’s also trivially easy to package as a docker image. What makes things like Zookeeper and Kafka to deploy is they are complex applications. You can write them in any other language and the deployment will still remain as non trivial.

I use Java for a living. Whether you think Java is very easy or a pain in the ass depends on what you're doing.

Consider the fact Ubuntu 18.04's default repos come with a pretty ancient version of Java. And if you want a newer version, you have to use some third-party source. And the fact there are both OpenJDK and Oracle JDKs to choose from. Will that be JDK or JRE? Headless? Depending on the combination of JDK and software, maybe the fonts will go all funny. Got multiple JDKs? Some programs will use your default Java, some will invite you to select the JDK to use, some will bundle their own complete JDK. Oh, a program uses JavaFX? No, of course that isn't installed just because you installed Java.

These problems aren't inherent to Java, of course - it's 98% due to Oracle's attempts to inconvenience people into paying for licenses.

Re: Kafka Removing Zookeeper Dependency

#82

Post by femiagbabiaka » Sun, May 17, 2020, 8:42 AM UTC

weego wrote: post #74

axaxs wrote: post #67

Earlier quoted context omitted.

This skips the steps of choosing and installing a JVM, which is a bit daunting. My distro gives like 4 different versions. And worse, some applications only run on certain versions. By default, there is no 'java' command.

That's not really a good faith argument. That's the equivalent of saying that a major problem with NodeJS is you have to pick which version to install before you deploy. I deploy single package jvm files most days having not had to consider the deploy environment for a about 4 years.

It is a good faith argument — it’s literally the whole point. Native binaries bypass all of that.

Re: Kafka Removing Zookeeper Dependency

#83

Post by jooz » Sun, May 17, 2020, 8:45 AM UTC

weego wrote: post #74

axaxs wrote: post #67

Earlier quoted context omitted.

This skips the steps of choosing and installing a JVM, which is a bit daunting. My distro gives like 4 different versions. And worse, some applications only run on certain versions. By default, there is no 'java' command.

That's not really a good faith argument. That's the equivalent of saying that a major problem with NodeJS is you have to pick which version to install before you deploy. I deploy single package jvm files most days having not had to consider the deploy environment for a about 4 years.

What is hard is not to choose the java version (1.8,19,...), but the JVM implementation. To choose between hotspot, openjdk, oracle. Even to find the download link without being forced to log in on oracle.com was difficult (in fact I couldnt find it the very last time).

Re: Kafka Removing Zookeeper Dependency

#84

Post by kirstenbirgit » Sun, May 17, 2020, 9:32 AM UTC

hugofirth wrote: post #4

I never know where I sit on stuff like this. On the one hand if you’re confluent I think it makes total sense to own this part of your infrastructure. Especially if it lets you improve your operability story. On the other hand I feel like projects should try and use open source “building-blocks”, like etcd and zookeeper, when building their distributed systems. Not only does this help iron out correctness bugs, but i…

I generally think it makes sense to start with readily available tools, but as a project grows and becomes more specialized, it often shows that the tool doesn't quite fit, doesn't quite have the same performance characteristics, or otherwise not a good fit anymore. And then it's time for something better. Or, it works out, and you don't switch it out (e.g. etcd in k8s)

Re: Kafka Removing Zookeeper Dependency

#85

Post by morelisp » Sun, May 17, 2020, 10:12 AM UTC

weego wrote: post #74

axaxs wrote: post #67

Earlier quoted context omitted.

This skips the steps of choosing and installing a JVM, which is a bit daunting. My distro gives like 4 different versions. And worse, some applications only run on certain versions. By default, there is no 'java' command.

That's not really a good faith argument. That's the equivalent of saying that a major problem with NodeJS is you have to pick which version to install before you deploy. I deploy single package jvm files most days having not had to consider the deploy environment for a about 4 years.

> That's the equivalent of saying that a major problem with NodeJS is you have to pick which version to install before you deploy.

Are you saying this isn't a major problem with Node? In my experience it's a problem with every language that requires a separate runtime.

The JVM is the worst of two worlds because I have the build complexity of an AOT compiler to make my deployment artifact and the deployment complexity of an interpreted language to get the correct runtime pre-installed.

Re: Kafka Removing Zookeeper Dependency

#86

Post by npstr » Sun, May 17, 2020, 10:21 AM UTC

jooz wrote: post #83

weego wrote: post #74

Earlier quoted context omitted.

That's not really a good faith argument. That's the equivalent of saying that a major problem with NodeJS is you have to pick which version to install before you deploy. I deploy single package jvm files most days having not had to consider the deploy environment for a about 4 years.

What is hard is not to choose the java version (1.8,19,...), but the JVM implementation. To choose between hotspot, openjdk, oracle. Even to find the download link without being forced to log in on oracle.com was difficult (in fact I couldnt find it the very last time).

Have a look at https://sdkman.io

Re: Kafka Removing Zookeeper Dependency

#87

Post by PopeDotNinja » Sun, May 17, 2020, 10:26 AM UTC

michaelt wrote: post #81

tiew9Vii wrote: post #73

Earlier quoted context omitted.

They are really simple if done correctly. You have a single fat jar so can do java -jar myapp.jar which is almost as easy as your static binary but prefixed with two extra words. It’s also trivially easy to package as a docker image. What makes things like Zookeeper and Kafka to deploy is they are complex applications. You can write them in any other language and the deployment will still remain as non trivial.

I use Java for a living. Whether you think Java is very easy or a pain in the ass depends on what you're doing. Consider the fact Ubuntu 18.04's default repos come with a pretty ancient version of Java. And if you want a newer version, you have to use some third-party source. And the fact there are both OpenJDK and Oracle JDKs to choose from. Will that be JDK or JRE? Headless? Depending on the combination of JDK and…

+1 for this. A headless install of Java is not straight forward. Doable? Yes. Pain in the butt? Yes.

Re: Kafka Removing Zookeeper Dependency

#88

Post by elric » Sun, May 17, 2020, 10:27 AM UTC

silisili wrote: post #59

elric wrote: post #48

Earlier quoted context omitted.

Things on the JVM are a PITA to deploy? That's a bit of a silly statement. Some things are easy to deploy, some aren't. It's not an intrinsic property of the JVM, but rather the consequence of a series of choices made by whoever wrote the tool in question. You can do TLS in Java without using keystores, even if keystores do have some advantages, and megacorps seem to like them for all the wrong reasons. Using them sh…

I don't understand Java apologists. I'm not saying Java is bad. Quite the opposite, it's one of the most optimized systems to date. But if you cannot admit Java apps are a pain to deploy, or, at least harder than deploying a static binary, I cannot take this argument seriously in good faith.

Seriously? When was the last time you deployed something as a standalone, static binary? Hardly anything with any amount of complexity is shipped like that, especially not cross-platform. Just about everything requires some kind of runtime, some kind of dependencies or some kind of environment. I can go "dnf install firefox" just as easily as I can go "dnf install ant". Both of these come with a boatload of dependencies. One of them depends on a JVM, the other doesn't. Both are packaged pretty well and both can be installed in a single step.

I indeed won't admit that "Java apps" are a pain to deploy, because it's a meaningless and overly generalistic statement. Some apps are a pain to deploy, but it's almost always down to the way they're packaged/distributed, and rarely down to the underlying technology.

Re: Kafka Removing Zookeeper Dependency

#89

Post by stingraycharles » Sun, May 17, 2020, 10:34 AM UTC

silisili wrote: post #59

elric wrote: post #48

Earlier quoted context omitted.

Things on the JVM are a PITA to deploy? That's a bit of a silly statement. Some things are easy to deploy, some aren't. It's not an intrinsic property of the JVM, but rather the consequence of a series of choices made by whoever wrote the tool in question. You can do TLS in Java without using keystores, even if keystores do have some advantages, and megacorps seem to like them for all the wrong reasons. Using them sh…

I don't understand Java apologists. I'm not saying Java is bad. Quite the opposite, it's one of the most optimized systems to date. But if you cannot admit Java apps are a pain to deploy, or, at least harder than deploying a static binary, I cannot take this argument seriously in good faith.

The problem I have with your statement is that you do not elaborate on why it is such a PITA. Maybe you had some issues with a Spring web app? JavaBeans? I don’t know. The problem is that you’re making a broad generalization about all apps on the JVM, while to me it seems like this has more to do with decisions of the specific apps you were deploying.

To go back to the original argument, why is Kafka ported from JVM to Go suddenly much easier to deploy? Is it really due to the JVM, or is it perhaps related to design decisions made while porting the application to Go?

Re: Kafka Removing Zookeeper Dependency

#90

Post by elric » Sun, May 17, 2020, 10:37 AM UTC

michaelt wrote: post #81

tiew9Vii wrote: post #73

Earlier quoted context omitted.

They are really simple if done correctly. You have a single fat jar so can do java -jar myapp.jar which is almost as easy as your static binary but prefixed with two extra words. It’s also trivially easy to package as a docker image. What makes things like Zookeeper and Kafka to deploy is they are complex applications. You can write them in any other language and the deployment will still remain as non trivial.

I use Java for a living. Whether you think Java is very easy or a pain in the ass depends on what you're doing. Consider the fact Ubuntu 18.04's default repos come with a pretty ancient version of Java. And if you want a newer version, you have to use some third-party source. And the fact there are both OpenJDK and Oracle JDKs to choose from. Will that be JDK or JRE? Headless? Depending on the combination of JDK and…

I don't know which JVMs are or aren't in Ubuntu. But Ubuntu 18 is over 2 years old. Almost every major linux distribution has stable, LTS versions of Java 8 and 11 -- which are the currently supported LTS versions in general.

A package can easily add a dependency on a specific Java version. And unlike many other tools, if you have multiple versions of Java, all you usually need to do is add the right one to the $PATH. Sometimes you may have to set $JRE_HOME. And that's it. You're done. If that's too much of a pain to deal with, I hope you never have to install a tool that requires a specific PHP, Python or NodeJS version.

I don't know of any serious tools written in Java that don't either explicitly say which version(s) are supported. Maybe some tools are poorly packaged. But again, that's not a problem with Java.