Replacing Kafka with SQLite?

#1

Post by mjirv » Wed, Aug 13, 2025, 11:49 PM UTC

Re: Replacing Kafka with SQLite?

#2

Post by 8organicbits » Thu, Aug 14, 2025, 1:09 AM UTC

> each Kafka partition was represented by a separate SQLite database

What does the programming interface look like for this? I believe you'd only be able to use SQL to query each partition (database) individually, unless there's another layer orchestrating SQL over the partitions. I wonder if you just end up with connectors again.

Re: Replacing Kafka with SQLite?

#3

Post by compressedgas » Mon, Aug 18, 2025, 5:13 AM UTC

8organicbits wrote: post #2

> each Kafka partition was represented by a separate SQLite database What does the programming interface look like for this? I believe you'd only be able to use SQL to query each partition (database) individually, unless there's another layer orchestrating SQL over the partitions. I wonder if you just end up with connectors again.

Additional databases can be added to a SQLite connection using:

  ATTACH 'pathname.db' AS prefix

https://www.sqlite.org/lang_attach.html