Viewing profile — hundredwatt

> To be pedantic, not guaranteed. The xor of multiple elements may erroneously have a passing checksum, resulting in an undetected false decode The false decodes can be detected. D…

The graph constructed by using bloom filter-style hash functions supports a decoding process called "peeling" where you: 1. Find a batch with 1 missing element 2. Delete that eleme…

You don't lose absolute guarantees, but the probabilistic nature means the process may fail (in a guaranteed detectable way) in which case you can try again with a larger parameter…

A neat trick to make the accumulator both collision-resistant and self-diagnosing. For every normalized link id x: y = (x If acc is zero, all links are reciprocal (same guarantee a…

The recently released sqlite_rsync utility uses a version of the rsync algorithm optimized to work on the internal structure of a SQLite database. It compares the internal data pag…

It’s meant to run in or alongside the pipeline continuously. Not planning to open source, working on a commercial offering but haven’t launched anything publicly yet. Would love to…

I'm building a new tool for end-to-end data validation and reconciliation in ELT pipelines, especially for teams replicating data from relational databases (Postgres, MySQL, SQL Se…

+100000

That’s one of the cases where query-based CDC may out perform log-based (as long as you don’t care to see every intermediate change that happened to a row between syncs)

Tuki is a small, open-source utility that lets you run one-off commands in production through Git, bringing version control and code review to production scripts. I built it after …