Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: UI for Apache Kafka

github.com

Post reply on HN
Print view View original

51–60 of 60 posts

Re: Show HN: UI for Apache Kafka

#51

Post by weeco »

mrweasel wrote: post #18
qrpike wrote: post #15

What is the difference between this and https://github.com/yahoo/CMAK

There's a lot great and developing tools for Kafka right now. CMAK doesn't seem that active, and I believe it was one of the tools we quickly gave up on, because it was a little difficult to get going. Currently we're using Kowl, and we're looking into Cruise Control. One limiting factor is that many of these tools doesn't support authentication and RBAC, Kowl does, and that has been a requirement for at least some o…

I'm curious what makes you look into other solutions than Kowl then? :). We are happy to add anything that is desired in the community. Now that Kafka connect support is also there we will most likely just expand the existing features (e.g. adding, editing schemas etc.).

Disclaimer: I'm one of the Kowl authors. I do not intend to hijack that post for Kowl content though!

Re: Show HN: UI for Apache Kafka

#52

Post by weeco »

spenczar5 wrote: post #8
germanosin wrote: post #6

Earlier quoted context omitted.

Btw I have an article comparing it with confluent web GUI https://towardsdatascience.com/overview-of-ui-tools-for-moni...

You should have mentioned that you work at Provectus in this article. Feels pretty sneaky to claim to do an impartial review, but provide no negative critique of the one made by your company…

This is indeed a very biased comparision where some features in the comparision table are just claimed to be not supported by other (open source) solutions. Even after reaching out multiple times the article is not updated accordingly. So this has to be considered a sneaky marketing page to be honest.

I might be biased as I'm one of the Kowl authors but I can definetely tell that Lenses is affected negatively the most. Imo it is one of the most complete paid solutions and got the worst marks basically.

Re: Show HN: UI for Apache Kafka

#53

Post by lrobinovitch »

PhilippGille wrote: post #31

There are several alternatives posted here already, but after comparing several ones maybe a year ago I found Kowl very nice: Nice UI, backend written in Go (quick startup, low memory usage), viewing and filtering of messages, support for schema registry, comes with a Docker image, usable locally and Helm chart for Kubernetes. https://github.com/cloudhut/kowl

Plus one for kowl. Their support is also amazing.

Re: Show HN: UI for Apache Kafka

#54

Post by juleska »

barbarbar wrote: post #33
vosper wrote: post #26

Earlier quoted context omitted.

> corporate memphis Links for the lazy. The GP page ( https://lenses.io ) certainly has a lot of good examples. https://en.wikipedia.org/wiki/Corporate_Memphis https://t-artmagazine.com/what-is-corporate-memphis-and-why-...

Haha thank you. Had no idea.

Me too hahaha

Re: Show HN: UI for Apache Kafka

#55

Post by mrweasel »

weeco wrote: post #51
mrweasel wrote: post #18

Earlier quoted context omitted.

There's a lot great and developing tools for Kafka right now. CMAK doesn't seem that active, and I believe it was one of the tools we quickly gave up on, because it was a little difficult to get going. Currently we're using Kowl, and we're looking into Cruise Control. One limiting factor is that many of these tools doesn't support authentication and RBAC, Kowl does, and that has been a requirement for at least some o…

I'm curious what makes you look into other solutions than Kowl then? :). We are happy to add anything that is desired in the community. Now that Kafka connect support is also there we will most likely just expand the existing features (e.g. adding, editing schemas etc.). Disclaimer: I'm one of the Kowl authors. I do not intend to hijack that post for Kowl content though!

For ourselves, nothing really. For a customer: They really don't want to pay the licens once it is required, and they need ActiveDirectory authentication or KeyCloak. I've already been in contact with someone working on Kowl, and we're just waiting on OIDC support.

Really we just want an alternative to Kowl, if a customer asks for suggestions. For our own use, we plan on keeping Kowl.

Re: Show HN: UI for Apache Kafka

#56

Post by germanosin »

daniel-grigg wrote: post #7

Neat! Just be wary around licensing, we’ve been using a similar management tool for a couple years https://github.com/yahoo/CMAK which was originally Kafka manager but they were forced to rename it.

We had a discussion with Apache team member according the name. Their main concern was that name should contain not Kafka but Apache Kafka. So they are ok with our current product name "UI for Apache Kafka"

Re: Show HN: UI for Apache Kafka

#58

Post by germanosin »

db3pt0 wrote: post #32

A few pieces of feedback. Positive: - Multi-cluster support is a great feature! That's the main thing missing from Kowl, in my opinion - The UI feels clean - Message streaming feels smooth, and the search feature is a great addition. Also love the ability to see a message for a given partition/offset, and that JSON messages are nicely rendered as JSON! Negative/Feature Requests: - The CleanupPolicy logic doesn't hand…

  Hi, db3pt0
  Thanks for detailed feedback!

  - The CleanupPolicy
  fixed issue 925 (https://github.com/provectus/kafka-ui/issues/925)

  - When viewing Cluster -> Topic -> Consumers, it seems like far more is being loaded than just consumer groups for that particular topic. 
  It takes much longer to load than, e.g., Kowl does for pulling the same information
  There is only one way to filter consumer groups by topic, get all consumer groups, enrich it with current members, committed offsets, and then filter them by topic.
  If you have a lot of consumer groups, this might take time to do it. We are thinking on this issue, and will try to improve it in the next versions.
  (https://github.com/provectus/kafka-ui/issues/927)

  - Similar feedback for when clicking on an individual consumer group (example URL path: "/ui/clusters/dev/consumer-groups/my-group"). It takes a very long time to load
 This looks rather strange. For single consumer we are getting consumer group description and then enrich it with topic info (end offsets). This works fast enough.

  - If any permissions issues are encountered while loading the individual consumer group, the entire request fails. That wouldn't necessarily be an issue, but it is when you're loading all consumer groups, and not just the one requested
  Thx for describing this. We created issue for this. (https://github.com/provectus/kafka-ui/issues/928)


  - I don't see any docs on how to access a topic that is secured with certificates (Kowl's relevant config [2])
  You have to pass ssl config stores to kafka client (we'll add docs. in the next versions https://github.com/provectus/kafka-ui/issues/929)

  KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_LOCATION = /var/private/ssl/kafka.server.keystore.jks
  KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_PASSWORD = test1234
  KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEY_PASSWORD = test1234
  KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_LOCATION = /var/private/ssl/kafka.server.truststore.jks
  KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_PASSWORD = test1234

  - If a schema registry uses a self-signed certificate (or one signed by a corporation's CA), there's no way to pass the certificate through a config or skip validation (Kowl's relevant config [3]). As it is, you get a 500 error in the API call, but the UI doesn't show an error

  Same as above (https://github.com/provectus/kafka-ui/issues/930)
  KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_LOCATION = /var/private/ssl/kafka.server.truststore.jks
  KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_PASSWORD = test1234

Re: Show HN: UI for Apache Kafka

#59

Post by germanosin »

mrweasel wrote: post #55
weeco wrote: post #51

Earlier quoted context omitted.

I'm curious what makes you look into other solutions than Kowl then? :). We are happy to add anything that is desired in the community. Now that Kafka connect support is also there we will most likely just expand the existing features (e.g. adding, editing schemas etc.). Disclaimer: I'm one of the Kowl authors. I do not intend to hijack that post for Kowl content though!

For ourselves, nothing really. For a customer: They really don't want to pay the licens once it is required, and they need ActiveDirectory authentication or KeyCloak. I've already been in contact with someone working on Kowl, and we're just waiting on OIDC support. Really we just want an alternative to Kowl, if a customer asks for suggestions. For our own use, we plan on keeping Kowl.

UI for Apache Kafka supports OIDC (https://github.com/provectus/kafka-ui/blob/master/guides/SSO...) and basic authentication (https://github.com/provectus/kafka-ui/blob/683ab231f436893b3...)

Re: Show HN: UI for Apache Kafka

#60

Post by germanosin »

dillon wrote: post #29

Where I work we use Confluent Cloud who has their own proprietary UI. I've always found it to be lacking, hard to use and not very good. We substituted it with AKHQ https://akhq.io/ , which is miles ahead of anything I've seen. The main issue with it are the interesting UX decisions that requires learning. For example, a lot of links require a double click, which isn't a common behavior in Web Apps. Besides that, it'…

UI for Apache also provides full support for Kafka Connect
Post reply on HN