Earlier quoted context omitted.
Precisely this! I might even add a section in the docs, “Not a replacement for Kafka,” under the FAQ. Where Kafka starts to fall short is routing. If you want to access the data of one user from user-events-topic, that’s expensive to do. Most other streaming technologies are built around the same design, such as Kinesis. There are other implementations that support the Kafka wire protocol and are cheaper in exchange…
Also (in case this isn't obvious) I'm 100% a fan of the decision to _not_ tightly couple to the kafka wire protocol. Kafka's apis are full of landmines and gotchas, and you can do much better from a UX perspective if you're not married to their quirky semantics. I had heard people talk about the operation pain involved in keeping Kafka alive (which is a thing for sure), but what I was surprised by was how many things…
But because the Pico's semantics are close to Kafka, but not tightly coupled, it's quite feasible to support the Kafka protocol, * with some restrictions *, such as a topic can only ever have one partition, no support for transactions (since they would only really apply when producers are publishing to multiple streams), and more along those lines. As a result, you'd get Kafka with no topic tax and a lot less operational complexity.