Viewing profile — galeaspablo

Let’s separate two advantages in the average case with Kafka. 1. Sequential IO when reading from disk. 2. Use of disk cache (instead of reading from disk) when re-reading recently …

No problem. :) Yes, order needs to be known. So no, it’s not possible to do O(w+h) with streams partitioned by key. Unless, of course you use a supplementary index, but then you mi…

Option 1, but after so many years banging our heads against the wall reasoning about this, we hoped someone would eventually give us a queue that supports arbitrary causal dependen…

Great intuition. Exactly right. More details in one of my other comments, here: https://news.ycombinator.com/item?id=43796688

Great follow up question, thank you. I could talk about this "topic" for days, so I appreciate the opportunity to expand. :) Let's imagine ourselves as a couple of engineers at Acm…

Apologies, we've been so deep into this problem that we take our slang for granted :) A graphical representation might be worth a thousand words, keeping in mind it's just one exam…

Some input from previously working on a superset of this problem. And being in a similar position. Mature projects have too much bureacracy, and even spending time talking to you =…

Right on, great question. Some examples: Example Option 1 You give up on the guarantees across partition keys (bank accounts), and you accept that balances will not reflect a causa…

That's correct. And keep in mind that you might have new consumers starting from the beginning come into play, so you have to permanently retain the indexes. And yes, O(n log n ) i…

Disclosure (given this is from Confluent): I'm ex MSK (Managed Streaming for Kafka at AWS) and my current company was competing with Confluent before we pivoted. Yup, this is one m…

It is useful, but it is not generally applicable. Given an arbitrary causality graph between n messages, it would be ideal if you could consume your messages in topological order. …

Agreed. The head of line problem is worth solving for certain use cases. But today, all streaming systems (or workarounds) with per message key acknowledgements incur O(n^2) costs …

Thanks for posting the NPR source. I read about the anesthesia case from VOX, and they claimed the opposite, namely that there would be no additional costs for the patients https:/…

Good leadership is spot on! Agreed. The cynic part of me sees incentives that discourage mature leadership styles. Leaders tend to be impatient and think of this quarter’s OKRs as …

Many engineers don’t truly care about the correctness issue, until it’s too late. Similar to security. Or they care but don’t bother checking whether what they’re doing is correct.…

If anything, even if he’s aware, I’d say it’s pretty much in line with the spirit. Albeit, I can’t speak for Tim O’Reilly, but just like I don’t expect to become a billionaire from…

I can’t help but read this under the lens of my own experiences. For me open source is really about capturing less than I create, much like it’s written in the article. Heck, I’m a…

Yes, for operational use cases. Eg event driven microservices communication. Keeping in mind we replace the sink as well, which allows us to do cool things such as https://ambar.cl…

Hey Denis, I haven’t run into you before. But hi, this is Luis, Ambar’s founder. Nice to meet a fellow data streamer. When I started writing Ambar I thought streaming from a databa…

We use Kafka under the hood. We stream instead of poll. We used to work at the Kafka team in AWS :) Our thesis is that a big blocker is the PhD the whole team needs in Kafka. For e…