Viewing profile — vira28

Reads like an add to Omnigent or whatever harness (wait it’s meta harness?.

Is that the core/tech behind Kitesurf?

TL;DR Initial bet didn’t work out and it also became irrelevant. Company is focusing on computer for agents. Founder CEO replaced with a new CEO.

I am learning about the Data eco-system and keep hearing about Icerberg as the open table format. However, I haven't come across anyone so far thats using it. Curious, if you see, …

Building https://github.com/viggy28/streambed - a full Postgres running on S3 using DuckDB as the query engine.

Certainly interested.

[dead]

Agree. Appreciate the reminder.

Not a hiring post. I have a 9-5 but also a lot of ideas I want to build. The core is just to show up and make small progress — but I find I'm missing accountability and partnership…

Tomas Vondra, a major Postgres contributor recently revived a thread on using Bloom filters - https://www.postgresql.org/message-id/flat/5cd8c20c-14b5-4b0... So there is more core …

The idea is pretty similar. As per their README, Iceberg support is deprecated.

From what I understand Ducklake needs a dedicated metadata database and it also ties to DuckDB land wherease with Iceberg many engines can query directly.

The challenge with any CDC is making it reliable. Curious, how are you exporting to S3? - Debezium or some service in AWS or home grown tool?

Currently, Strembed expects REPLICA IDENTITY FULL for getting the before and after value of TOAST column. Since we have the data in object storage, we could populate it without the…

Welcome. To avoid primary running out of disk space, you can configure max_slot_wal_keep_size https://www.postgresql.org/docs/17/runtime-config-replicatio... Since Supabase is vani…

Aside from the cost, my major motivation is to keep the infrastructure simple. The data is already there in Postgres, so I didn't want to add another data warehouse. I have also sh…

Thanks for the kind words! Short answer: yes, column-level schema changes sync to Iceberg automatically[0]. Logical replication (pgoutput in v1) doesn't actually stream DDL stateme…

[flagged]

Welcome. Would love to hear your experience. Feel free to share here or in the repo. Fully open source.

Should be fairly doable using binlog-based producer https://github.com/go-mysql-org/go-mysql .

Both projects are relevant. Curious, what kinda pushdown capabilities that you were looking for?

Hello, I checked ingestr repo, and it is in my bookmark. Small world. Agree, CDC is like Death by a thousand cuts. I believe Debezium has a Java library. My initial need was Postgr…

[flagged]