I see no downside in using sqlite as an application file format.
SQLite as an Application File Format
31–40 of 108 posts
Re: SQLite as an Application File Format
#32Bit unrelated rant but I'm still not sure why ZIP has been adopted as an Application File Format rather than anything else. It is a remanent of a DOS era with questionable choices, why would you pick it over anything else?
- archiver format to stow multiple files in one; your actual files (in your choice of format(s)) go inside - files can be individually extracted, in any order, from the archive - thousands of implementations available, in every language and every architecture. no more than 32KiB RAM needed for decompression - absolutely no possibility of patent challenges
Also architecturally suitable for the common case of collecting heterogeneous files in existing and new formats into a single file, as opposed to designing a database schema or a complex container structure from scratch.
Any multi-file archive format would do, but ZIP is very portable and random access.
Re: SQLite as an Application File Format
#33Somehow my first thought from the title was using sqlite as a format for applications. So like a replacement for ELF. I think this idea is both fascinating and horrifying.
Or a replacement for Access
Re: SQLite as an Application File Format
#34Earlier quoted context omitted.
I think most format use "gzip" instead of "zip".
gzip is not an archive container. You're thinking of .tar.gz which is a "tape archive" format which is compressed using gzip. Zip is by itself both a compression and an archive format, and is what documents like epub or docx use
You are right, but other documents like .ggb (GeoGebra files) or .mbz (Moodle backups) use the .tar.gz method. I even wrote programs to opened them, make a few tweaks and save the new version in another compatible file.
Re: SQLite as an Application File Format
#35Somehow my first thought from the title was using sqlite as a format for applications. So like a replacement for ELF. I think this idea is both fascinating and horrifying.
wonder if this would make hot-swap functions easier, if every function had its own section and every section was in the db
I think we could call it Library Internal Sequel Procedures.
Re: SQLite as an Application File Format
#36Bit unrelated rant but I'm still not sure why ZIP has been adopted as an Application File Format rather than anything else. It is a remanent of a DOS era with questionable choices, why would you pick it over anything else?
If all you need is a bag of named blobs and you just want quick reasonable compression supported across all platforms, why not?
If you don't need any table/relational data and are always happy to rewrite the entire file on every save, ZIP is a perfectly fine choice.
It's easier than e.g. a SQLite file with a bunch of individually gzipped blobs.
Re: SQLite as an Application File Format
#37This approach has really helped me out in my work. I do something very similar using DuckDB to slurp output files anytime I write a custom hierarchical model. The single sql queryable file simplified my storage and analytics pipeline. I imagine SQLite would be especially ideal where long term data preservation is critical.
Re: SQLite as an Application File Format
#38Something to consider when using SQLite as a file format is compression (correct me if I'm wrong!). You might end up with a large file unless you consider this, and can't/won't just gz the entire db. Nothing is compressed by default.
Re: SQLite as an Application File Format
#39Something to consider when using SQLite as a file format is compression (correct me if I'm wrong!). You might end up with a large file unless you consider this, and can't/won't just gz the entire db. Nothing is compressed by default.
It can be compressed, see https://sqlite.org/sqlar.html
Re: SQLite as an Application File Format
#40We are developing using sqlite to transfer configurations from uat to production environment. Since the configurations are already saved in a postgres table in uat, moving some configs from uat to production an sqlite file is very easy. since it's a binary format, we are also saved from any inadvertent edits by people doing production deployment.
Also, another usecase is to export data from production to uat for testing some scenarios, it can be easily encoded in a sqlite file.