[flagged]
Java APIs are indeed awfully complex.
More than that. It is a nightmare to make binding for. So if you want to really be extensible through plugins, supporting other languages is very important.
What If We Could Rebuild Kafka from Scratch?
21–30 of 229 posts
Re: What If We Could Rebuild Kafka from Scratch?
#22[flagged]
Why? It's fast, featureful, well maintained and there are tons of people who know the language?
Re: What If We Could Rebuild Kafka from Scratch?
#23Earlier quoted context omitted.
I'm wondering how much of that is bad developer UX and defaults, and how much of that is inherent complexity in the problem space. Like the article outlines, partitions are not that useful for most people. Instead of removing them, how about having them behind a feature flag, i.e. not on by default. That would ease 99% of users problems. The next point in the article which to me resonates is the lack of proper schema…
I think it's just horrible software built on great ideas sold on a false premise (this is a generic message queue and if you don't use this you cannot "scale").
[deleted]
Re: What If We Could Rebuild Kafka from Scratch?
#24Since we are dreaming - add ETL there as well!
Re: What If We Could Rebuild Kafka from Scratch?
#25I feel like everyone's journey with Kafka ends up being pretty similar. Initially, you think "oh, an append-only log that can scale, brilliant and simple" then you try it out and realize it is far, far, from being simple.
The worst part of Kafka, for me, is managing the cluster. I don't really like the partitioning and the almost hopelessness that ensues when something goes wrong. Recovery is really tricky.
Granted it doesn't happen often, if you plan correctly, but the possibility of going wrong in the partitioning and replication makes updates and upgrades nightmare fuel.
Re: What If We Could Rebuild Kafka from Scratch?
#26Interesting, if partitioning is not a useful concept of Kafka, what are some of the better alternatives for controlling consumer concurrency?
Re: What If We Could Rebuild Kafka from Scratch?
#27I know it’s not what the article is about but I really wish we could rebuild Franz Kafka and hear what he thought about the tech dystopia we are in.
>I cannot make you understand. I cannot make anyone understand what is happening inside me. I cannot even explain it to myself. -Franz Kafka, The Metamorphosis
Re: What If We Could Rebuild Kafka from Scratch?
#28[flagged]
> Step 1, stop using Java. I've seen these comments for over 15 years yet for some "unknown", "silly" reason java keeps being used for really,really useful software like kafka.
But it's also the reason for why these Apache projects systematically get displaced by better and faster C, C++, Rust or Go alternatives.
Re: What If We Could Rebuild Kafka from Scratch?
#29[dead]
Re: What If We Could Rebuild Kafka from Scratch?
#30Earlier quoted context omitted.
I'm wondering how much of that is bad developer UX and defaults, and how much of that is inherent complexity in the problem space. Like the article outlines, partitions are not that useful for most people. Instead of removing them, how about having them behind a feature flag, i.e. not on by default. That would ease 99% of users problems. The next point in the article which to me resonates is the lack of proper schema…
I think it's just horrible software built on great ideas sold on a false premise (this is a generic message queue and if you don't use this you cannot "scale").
I kind of agree on the horrible software bit, but what do you use instead? And can you convince your company to use that, too?