Any recommendations from HN for a write-once (literally once), data storage format that's suitable for network storage? sqlite docs recommend avoiding using it on network storage, though from what I can gather, it's less of an issue if you're truly only doing reads (meaning I could create it locally and then copy it to network storage). Apache Parquet seems promising, and it seems to support indexing now which is an…
SQLite's File Format
81–86 of 86 posts
Re: SQLite's File Format
#82My only question is if you really need a prefix before every value to say what type it is.
Any field in SQLite can contain any type, even if the schema says that a field should be INTEGER, it could have a TEXT, so it's necessary to specify what's the type of every single value
Re: SQLite's File Format
#83Earlier quoted context omitted.
I've had multiple flaky issues with SQLite (e.g. non-HA Grafana) on Azure Files using NFS v4.1 leading to locked DBs. Perhaps some implementations work, I'm not gonna rely on it or advise others to do so.
Yeah trying to write from several hosts will certainly fail if you don't have advisory locks working, which is not a given, so you are right of course
Re: SQLite's File Format
#84Earlier quoted context omitted.
>> It was designed to be a DB for Tcl at a time when that language didn't have typed objects. Its SQL implementation reflects that. > It actually does have typed values Now. As the article points out, they were not part of the initial design, because of the Tcl heritage.
AFAIK it has always had typed values. Don’t confuse column types (which constrain a column to containing only values of a specified type) with value types (which enable it to treat the string “12” and the integer 12 and the floating point 12.0 as three distinct values) Tcl has value types. Tcl 7.x and earlier only had one data type, the string-so adding two integers required two string-to-int conversions followed by…
Re: SQLite's File Format
#85The fact this fits in a few pages and is so approachable is a testament to its simplicity. I think I'd find it a lot harder to grok the file format of, for example, a Word doc/docx file.
I wouldn't put .doc and .docx next to one another, as they're only tangentially related. I'd bet getting the hello, world of .docx would be some silliness, but would not be hard to grok. I couldn't readily find a browsable copy of ECMA 376 4th Ed online but https://github.com/PumasAI/WriteDocx.jl/blob/v1.2.0/docs/src... was in the ballpark of what I expected to find in some section of the actual spec
Re: SQLite's File Format
#86Earlier quoted context omitted.
AFAIK it has always had typed values. Don’t confuse column types (which constrain a column to containing only values of a specified type) with value types (which enable it to treat the string “12” and the integer 12 and the floating point 12.0 as three distinct values) Tcl has value types. Tcl 7.x and earlier only had one data type, the string-so adding two integers required two string-to-int conversions followed by…
Tcl still only has one data type, which is the string. It has different internal representations to allow representing integers etc more efficiently, but since that is not part of the formal semantics of the language, those representations are not data types.