[flagged]
Tracking down the 16-year-old WAL-reset SQLite bug
151–160 of 263 posts
Re: Tracking down the 16-year-old WAL-reset SQLite bug
#152Earlier quoted context omitted.
Discussion here: https://news.ycombinator.com/item?id=49277799
Wait, it's AI? AI probably has information about this bug by now, since it was discovered and fixed in March.
Nope, it's a generic PBT harness. The human who built it knew about the bug, but the workload was specifically designed not to specifically target this particular bug, and may have found several others as well (we are in the process of reporting these upstream).
Re: Tracking down the 16-year-old WAL-reset SQLite bug
#153Earlier quoted context omitted.
Yeah, this part also stuck out to me: > Because this wouldn’t be a quick or easy fix, we reached out to the SQLite developers for a professional support contract. This was a great decision. It gave us direct access to their deep expertise and experience, and we had many detailed technical conversations about our architecture and our incidents. They were willing to pay to get help solving the problem, and then pay aga…
They could have just spent $300,000 in tokens to AI slop rewrite SQLite in Rust.
That would be lame. Instead they spent less and improved the world for everyone who used SQLite (which is really a lot of people)
Re: Tracking down the 16-year-old WAL-reset SQLite bug
#154This reaffirms my belief that SQLite is not well suited for systems with significant concurrency. It replaces fopen, not postgres. Although this corruption is a rare bug and sqlite is usually extremely stable, it's usually not worth it from a performance and features standpoint either. Here they were trying to do a backup by forcing a checkpoint and then copying the file. Systems like postgres let you do online conti…
Yeah, I also feel 'just use SQLite [no matter what]' is just the pendulum swinging hard after the 'just use mongodb [no matter what]' of yesteryear. It's so sometimes just performative. I remember when Tailscale had a similar performative approach with 'just use a JSON file on disk'. Then etcd. Then SQLite. Like sure, you can keep picking the absolite mininum technology for your needs and then change it every couple…
Postgres has also had pretty serious bugs, eg fsyncgate
Re: Tracking down the 16-year-old WAL-reset SQLite bug
#155I wonder if this also affected litestream disproportionately, because litestream also inserts itself into the checkpoint process.
I would assume they actually moved off of litestream here no? otherwise how can their frequent manual checkpointing even succeed when litestream locks for the same behavior.
I meant does the rare SQLite bug tend to manifest in litestream for the same reason it showed up in Tailscale.
Re: Tracking down the 16-year-old WAL-reset SQLite bug
#156The irony is that the SQLite developers get a support contract iff someone runs off the path in anger and finds an ancient bug. But perhaps that's part of what make it a quality team: devotion thriving without adverse incentives.
While you might be correct, I wouldn't necessarily assume the "and only if" part of your statement. There might be companies that choose to proactively purchase support contracts. And there are companies that have paid $150K/year for https://sqlite.org/consortium.html access. Which means, among other things, that they get first priority for any needs they have:
> Consortium members have the guaranteed, undivided attention of the SQLite developers for 23 staff-days per year and for as much additional time above and beyond that amount that the core developers have available. There are no arbitrary limits on contact time. The consortium will never be over-subscribed. New SQLite developers will be recruited and trained as necessary to cover the 23 day/year support commitment.
The SQLite home page lists five companies that have paid for consortium access. I can easily imagine that there are more who don't want to pay $150K/year but would pay $1.5k/year, proactively, to get "private, expert email advice from the developers of SQLite" when they need it.
Re: Tracking down the 16-year-old WAL-reset SQLite bug
#157Earlier quoted context omitted.
It can prove absence of specific bugs though.
no it can't. what if this bug is still there but the timing window is now one CPU instruction?
Then it’s a different bug: they fixed a bug with a wider timing window and introduced a new bug with a narrower timing window.
Re: Tracking down the 16-year-old WAL-reset SQLite bug
#158Earlier quoted context omitted.
Tailscale is the best. It's infinitely better than Hamachi, ZeroTier, etc. My only gripe is that they have some really weird SSO requirements like GitHub, etc. and then that provider becomes a permanent part of your identity.
I've been a fan of Tailscale since encountering it for the first time at a previous job at a small startup. Someone asked if anyone had a Linux machine when we were all testing out something, and I mentioned I had a personal Linux desktop but wasn't sure how to connect it to the VPN for access, and it turned out that the solution was just literally running two commands in the terminal after installing tailscale from…
That strongswan thing is the kind of design HN praises about open protocols by IRC (just the other day...) but in practice is so flexible it can't keep itself upright and it's unusable in practice.
Meanwhile tailscale or wireguard, by being actually opinionated, avoids needing much configuration at both ends.
Re: Tracking down the 16-year-old WAL-reset SQLite bug
#159Earlier quoted context omitted.
no it can't. what if this bug is still there but the timing window is now one CPU instruction?
> what if this bug is still there but the timing window is now one CPU instruction? I think a model checker or something similar that can exhaustively cover a search space would suffice, though at that point I think the boundary between "test" and "formal verification" becomes somewhat fuzzy.
So, a proof, not a test.
Re: Tracking down the 16-year-old WAL-reset SQLite bug
#160Excellently-written article. Bravo!