SQLite: Wal2 Mode Notes
sqlite.org
SQLite: Wal2 Mode Notes
1–10 of 16 posts
Re: SQLite: Wal2 Mode Notes
#2SQLite has different “modes” to facilitate recovery, this new WAL2 mode addresses the problem in WAL (version 1) where the recovery log could potentially grow very large.
It’s solving a real problem, but not considered stable for production yet.
Re: SQLite: Wal2 Mode Notes
#3SQLite has different “modes” to facilitate recovery, this new WAL2 mode addresses the problem in WAL (version 1) where the recovery log could potentially grow very large. It’s solving a real problem, but not considered stable for production yet.
Citation needed.
Re: SQLite: Wal2 Mode Notes
#4SQLite has different “modes” to facilitate recovery, this new WAL2 mode addresses the problem in WAL (version 1) where the recovery log could potentially grow very large. It’s solving a real problem, but not considered stable for production yet.
I am curious what the situation is where the recovery log grows to large size and what the actual consequence of this would be.
We have been using SQLite in WAL mode for over half a decade and never witnessed this. Several of our databases can see concurrent access from hundreds of users with transactions in the 1-10 megabyte range, so I find it a bit odd this never came up.
Re: SQLite: Wal2 Mode Notes
#5SQLite has different “modes” to facilitate recovery, this new WAL2 mode addresses the problem in WAL (version 1) where the recovery log could potentially grow very large. It’s solving a real problem, but not considered stable for production yet.
Citation needed.
For which claim?
The recovery log growing large is explained in [1] - "it does mean that the wal file may grow indefinitely [...] There are also circumstances [...] causing the wal file to grow indefinitely in a busy system."
Not ready for production is [2] - although that is from 2 years ago, the WAL2 branch still isn't merged to trunk [3] which you'd expect if it was ready to go, I think?
[1] https://www.sqlite.org/cgi/src/doc/wal2/doc/wal2.md [2] https://sqlite.org/forum/forumpost/17249fb83a?t=c&unf [3] https://www.sqlite.org/cgi/src/timeline?r=wal2
Re: SQLite: Wal2 Mode Notes
#6SQLite has different “modes” to facilitate recovery, this new WAL2 mode addresses the problem in WAL (version 1) where the recovery log could potentially grow very large. It’s solving a real problem, but not considered stable for production yet.
I am curious what the situation is where the recovery log grows to large size and what the actual consequence of this would be. We have been using SQLite in WAL mode for over half a decade and never witnessed this. Several of our databases can see concurrent access from hundreds of users with transactions in the 1-10 megabyte range, so I find it a bit odd this never came up.
> I am curious what the situation is where the recovery log grows to large size
From the link, "if a writer writes to the database while a checkpoint is ongoing [...] it does mean that the wal file may grow indefinitely if the checkpointer never gets a chance to finish without a writer appending to the wal file. There are also circumstances in which long-running readers may prevent a checkpointer from checkpointing the entire wal file - also causing the wal file to grow indefinitely in a busy system."
Re: SQLite: Wal2 Mode Notes
#7I wonder why they limit it to 2 WAL files instead of creating a new WAL file once the latest one reaches a certain size and a checkpoint is done. Then delete the WAL files that have been successfully flushed to the main DB. That could minimize the spikyness of WAL file garbage collection under less than optimal circumstances.
Re: SQLite: Wal2 Mode Notes
#8SQLite has different “modes” to facilitate recovery, this new WAL2 mode addresses the problem in WAL (version 1) where the recovery log could potentially grow very large. It’s solving a real problem, but not considered stable for production yet.
I am curious what the situation is where the recovery log grows to large size and what the actual consequence of this would be. We have been using SQLite in WAL mode for over half a decade and never witnessed this. Several of our databases can see concurrent access from hundreds of users with transactions in the 1-10 megabyte range, so I find it a bit odd this never came up.
I ran into this when I was importing about 1TB into a DB while simultaneously reading from it and performing tasks. This was all in a dev environment, but it did come up.
Re: SQLite: Wal2 Mode Notes
#9I wonder why they limit it to 2 WAL files instead of creating a new WAL file once the latest one reaches a certain size and a checkpoint is done. Then delete the WAL files that have been successfully flushed to the main DB. That could minimize the spikyness of WAL file garbage collection under less than optimal circumstances.
As an informed guess: backward compatibility.
This way an SQLite library which has never heard of a WAL2 file will just use WAL mode instead, instead of potentially being confused by the unheard-of existence of multiple .wal files.
Re: SQLite: Wal2 Mode Notes
#10SQLite has different “modes” to facilitate recovery, this new WAL2 mode addresses the problem in WAL (version 1) where the recovery log could potentially grow very large. It’s solving a real problem, but not considered stable for production yet.
I am curious what the situation is where the recovery log grows to large size and what the actual consequence of this would be. We have been using SQLite in WAL mode for over half a decade and never witnessed this. Several of our databases can see concurrent access from hundreds of users with transactions in the 1-10 megabyte range, so I find it a bit odd this never came up.
> what the actual consequence of this would be.
SQLite runs in a surprising number of places. In an embedded environment disk space may be limited.