Breaking the WAL

#1

Post by wwilson » Wed, Aug 12, 2026, 8:00 PM UTC

Re: Breaking the WAL

#2

Post by john_strinlai » Wed, Aug 12, 2026, 8:40 PM UTC

related: Tailscale Traces Database Corruption to 16y/o SQLite WAL-Reset Bug (tailscale.com) https://news.ycombinator.com/item?id=49272832

Re: Breaking the WAL

#3

Post by wwilson » Wed, Aug 12, 2026, 8:42 PM UTC

We wanted to publish this ASAP since people are talking about this bug today. In a follow-up, we will show how our automated causality analysis could have saved Tailscale and SQLite from 6 months of root cause analysis. Stay tuned!

Re: Breaking the WAL

#4

Post by grebc » Wed, Aug 12, 2026, 8:46 PM UTC

[flagged]

Re: Breaking the WAL

#5

Post by wwilson » Wed, Aug 12, 2026, 8:49 PM UTC

grebc wrote: post #4

[flagged]

As the post (and linked repo) pretty clearly indicate, zero info about the existence of the bug went into this work.

Oh btw, we also found some other ones… Stay tuned!

Re: Breaking the WAL

#6

Post by carlsverre » Wed, Aug 12, 2026, 8:50 PM UTC

grebc wrote: post #4

[flagged]

Carl (author) here. This is a poor reading of the blog post. Notably, I built a general-purpose workload that simply exercises the write/checkpoint machinery. The exact same workload we use to break all kinds of transactional systems. This generic workload found the bug without any special knowledge of the bug, due to how Antithesis works.

Re: Breaking the WAL

#7

Post by grebc » Wed, Aug 12, 2026, 8:51 PM UTC

wwilson wrote: post #5

grebc wrote: post #4

[flagged]

As the post (and linked repo) pretty clearly indicate, zero info about the existence of the bug went into this work. Oh btw, we also found some other ones… Stay tuned!

So you found this before SQLite published the fix?

Re: Breaking the WAL

#8

Post by grebc » Wed, Aug 12, 2026, 8:54 PM UTC

carlsverre wrote: post #6

grebc wrote: post #4

[flagged]

Carl (author) here. This is a poor reading of the blog post. Notably, I built a general-purpose workload that simply exercises the write/checkpoint machinery. The exact same workload we use to break all kinds of transactional systems. This generic workload found the bug without any special knowledge of the bug, due to how Antithesis works.

I read your ad.

It states you replicated the bug once the SQLite team fixed it, and published it.

Not sure what’s difficult about replicating behaviour when it’s spelled out for you.

Re: Breaking the WAL

#9

Post by carlsverre » Wed, Aug 12, 2026, 8:55 PM UTC

Hi! I'm the author and the person who was on a road trip when I decided to do this experiment back in July. I'm super excited to finally get the blog post out, and even better, alongside a post describing the root cause process that went into the project. You can read about Tailscale's story here: https://tailscale.com/blog/sqlite-wal-reset-bug

I'm only sad that I didn't put SQLite under test earlier in the year, or I would have found this issue right away. If you look at the workload[1], you can see how simple it is. Exactly the same kind of workload we write every day to help our customers find bugs.

If you have any questions about our process or how debugging with Antithesis works, please let me know! Thanks for reading!

[1]: https://github.com/antithesishq/sqlite/blob/3.51.2-instrumen...

Re: Breaking the WAL

#10

Post by biorach » Wed, Aug 12, 2026, 8:58 PM UTC

grebc wrote: post #8

carlsverre wrote: post #6

Earlier quoted context omitted.

Carl (author) here. This is a poor reading of the blog post. Notably, I built a general-purpose workload that simply exercises the write/checkpoint machinery. The exact same workload we use to break all kinds of transactional systems. This generic workload found the bug without any special knowledge of the bug, due to how Antithesis works.

I read your ad. It states you replicated the bug once the SQLite team fixed it, and published it. Not sure what’s difficult about replicating behaviour when it’s spelled out for you.

this is a deliberate mischaracterisation of the content of the article