To me it sounds like NATS Jetstream but with Rust. I wonder what the reliability looks like when it is prod ready
Small Kafka: Tansu and SQLite on a free t3.micro
11–20 of 27 posts
Re: Small Kafka: Tansu and SQLite on a free t3.micro
#12This SQLite obsession is getting quite ridiculous. Now they put it in "the Cloud." What a shitshow. I wonder whether they know what SQLite is for... when Cloudflare did it, well, it made sense at least. This new generation of SQLite caro-culting is beyond anything I've ever seen.
Re: Small Kafka: Tansu and SQLite on a free t3.micro
#13This SQLite obsession is getting quite ridiculous. Now they put it in "the Cloud." What a shitshow. I wonder whether they know what SQLite is for... when Cloudflare did it, well, it made sense at least. This new generation of SQLite caro-culting is beyond anything I've ever seen.
Tansu author here. Storage is a pluggable choice of: PostgreSQL, memory, SQLite or S3. There are others in the pipeline (SlateDB, ...).
Re: Small Kafka: Tansu and SQLite on a free t3.micro
#14Any good and honest tansu experience reports out there? Would be nice to understand how “bleeding edge” this actually is, in practice. The idea of a kafka compatible, but trivial to run, system like this is very intriguing!
I wonder how it compares to Redpanda
I've used Redpanda for local development and testing stands. It is super easy to setup in docker, starts really fast and consumes less resources than Java version. Haven't really compared it to anything, but I remember using Java version of Kafka before and it was a resource hog. It is important when you develop on laptop with constrained resources.
Re: Small Kafka: Tansu and SQLite on a free t3.micro
#15To me it sounds like NATS Jetstream but with Rust. I wonder what the reliability looks like when it is prod ready
Jetstream isn't kafka-compatible, nor does it have pluggable storage of s3, sqlite, Postgres etc...
Re: Small Kafka: Tansu and SQLite on a free t3.micro
#16Earlier quoted context omitted.
I wonder how it compares to Redpanda
I've used Redpanda for local development and testing stands. It is super easy to setup in docker, starts really fast and consumes less resources than Java version. Haven't really compared it to anything, but I remember using Java version of Kafka before and it was a resource hog. It is important when you develop on laptop with constrained resources.
What I meant was how Tensu compares to Redpanda
Re: Small Kafka: Tansu and SQLite on a free t3.micro
#17Earlier quoted context omitted.
I wonder how it compares to Redpanda
I've used Redpanda for local development and testing stands. It is super easy to setup in docker, starts really fast and consumes less resources than Java version. Haven't really compared it to anything, but I remember using Java version of Kafka before and it was a resource hog. It is important when you develop on laptop with constrained resources.
to be fair, Kafka now has a GraalVM docker image[0][1] which was made for local dev/testing, and it has caught up fairly well to these alternatives re: memory and startup time
[0] - https://cwiki.apache.org/confluence/display/KAFKA/KIP-974%3A... [1] - https://hub.docker.com/r/apache/kafka-native
Re: Small Kafka: Tansu and SQLite on a free t3.micro
#18Peter (the author) is a really, really cool guy. We recorded a 3hr 30m podcast[0] with him a month ago. For anyone interested in the Kafka space, performance optimization in Rust and the general "why yet another Kafka", I'd shamelessly recommend the video:
Re: Small Kafka: Tansu and SQLite on a free t3.micro
#19This SQLite obsession is getting quite ridiculous. Now they put it in "the Cloud." What a shitshow. I wonder whether they know what SQLite is for... when Cloudflare did it, well, it made sense at least. This new generation of SQLite caro-culting is beyond anything I've ever seen.
Tansu author here. Storage is a pluggable choice of: PostgreSQL, memory, SQLite or S3. There are others in the pipeline (SlateDB, ...).
Any chance of a Parquet compatible storage choice?
Re: Small Kafka: Tansu and SQLite on a free t3.micro
#20Earlier quoted context omitted.
Tansu author here. Storage is a pluggable choice of: PostgreSQL, memory, SQLite or S3. There are others in the pipeline (SlateDB, ...).
Any chance of a Parquet compatible storage choice?
Yes: with a schema backed topic (AVRO, JSON or Protocol buffer) Tansu can write to Apache Iceberg, Delta or Parquet. You can use a Sink topic to write directly to an open table format (including Parquet) skipping (most of) the Kafka metadata.