31–40 of 109 posts
As a document _exchange_/_interchange_ format, what I prefer for durability is a non-binary format (e.g. XML based).
For local use, I agree SQLite might be much faster than ZIP, and of course the ability to query based on SQL has its own flexibility merits.
wouldn’t an XML database be easier?
Didn’t Apple actually move to SQLite for their Pages/Numbers format? I remember reading years ago that it was rocky (the transition), but was maybe eventually smoothed out?
$ file concessions-stand-menu-template.pages
concessions-stand-menu-template.pages: Zip archive data, at least v2.0 to extract, compression method=store
$ unzip -l concessions-stand-menu-template.pages
Archive: concessions-stand-menu-template.pages
Length Date Time Name
--------- ---------- ----- ----
58727 05-09-2022 13:27 Data/Artboard 2-26.png
26993 05-09-2022 13:27 Data/Artboard 2-small-27.png
11550 05-10-2022 08:13 Index/Document.iwa
720 05-10-2022 08:13 Index/ViewState.iwa
536 05-09-2022 12:41 Index/CalculationEngine-1686619.iwa
23 07-02-2021 17:48 Index/AnnotationAuthorStorage-1686618.iwa
43891 05-09-2022 12:41 Index/DocumentStylesheet.iwa
229 05-09-2022 13:28 Index/DocumentMetadata.iwa
17895 05-10-2022 08:13 Index/Metadata.iwa
379 05-10-2022 08:13 Metadata/Properties.plist
36 05-09-2022 12:41 Metadata/DocumentIdentifier
268 04-29-2022 22:18 Metadata/BuildVersionHistory.plist
135503 05-10-2022 08:13 preview.jpg
1666 05-10-2022 08:13 preview-micro.jpg
11057 05-10-2022 08:13 preview-web.jpg
--------- -------
309473 15 files
$ unzip concessions-stand-menu-template.pages Index/Document.iwa
extracting: Index/Document.iwa
$ file Index/Document.iwa
Index/Document.iwa: data
$ xxd -l 128
00000000: 001a 2d00 bcae 0170 6408 0112 6008 904e ..-....pd...`..N
00000010: 1203 0100 0518 c90c 2209 0a03 0a01 3010 ........".....0.
00000020: 0118 0122 0701 0b08 2e18 0109 1400 2f05 ..."........../.
00000030: 14f4 a801 0b0a 050a 030f 0111 1003 1800 ................
00000040: 2a27 daf8 66db f866 dcf8 66e6 f768 ddf8 *'..f..f..f..h..
00000050: 66df ef66 def8 66d1 f666 fdf5 66d5 f566 f..f..f..f..f..f
00000060: 8ff8 66df f866 86f9 6612 0408 def8 661a ..f..f..f.....f.
00000070: 0408 fdf5 6622 0408 8ff8 6632 0408 dfef ....f"....f2....[1] https://github.com/lifthrasiir/angel/commit/50a15e703ef2c1af...
SQLite can't be reliably used in networked file systems because it heavily relies on locking to be correctly implemented. I recently had to add a check for such file systems in my application [1] because I noticed a related corruption firsthand. Simpler file formats do not demand such requirements. SQLite is certainly good, but not for this use. [1] https://github.com/lifthrasiir/angel/commit/50a15e703ef2c1af...
SQLite can't be reliably used in networked file systems because it heavily relies on locking to be correctly implemented. I recently had to add a check for such file systems in my application [1] because I noticed a related corruption firsthand. Simpler file formats do not demand such requirements. SQLite is certainly good, but not for this use. [1] https://github.com/lifthrasiir/angel/commit/50a15e703ef2c1af...
Easy fix is an empty lock file adjacent to the real one.
wouldn’t an XML database be easier?
If you're going to use SQLite as an application file format, you should: 1. Enable the secure_delete pragma https://antonz.org/sqlite-secure-delete/ > so that when your user deletes something, the data is actually erased. Otherwise, when a user shares one of your application's files with someone else, the recipient could recover information that the sender thought they had deleted. 2. Enable the options described at…
If you're going to use SQLite as an application file format, you should: 1. Enable the secure_delete pragma https://antonz.org/sqlite-secure-delete/ > so that when your user deletes something, the data is actually erased. Otherwise, when a user shares one of your application's files with someone else, the recipient could recover information that the sender thought they had deleted. 2. Enable the options described at…
Earlier quoted context omitted.
Easy fix is an empty lock file adjacent to the real one.
Yeah, but only if SQLite did support that mode in some built-in VFS implementation...
Sqlite on NFSv3 has been rock solid for some NFS servers for a decade.
Maybe name and shame?