Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if OpenDocument used SQLite? (2014)

sqlite.org

Post reply on HN
Print view View original

251–260 of 310 posts

Re: What if OpenDocument used SQLite? (2014)

#251

Post by sdeframond »

sdeframond wrote: post #246
bane wrote: post #157

Earlier quoted context omitted.

I'm not sure if the problem you are pointing out has to do with: a) SQLite the file format - which is Public Domain and so well documented that parsers for it exist in numerous other languages even though it's almost pointless because... b) SQLite, the Public Domain (and thus entirely source available) C implementation of the library that can operate on the file format -- and is documented to a level well above what…

I think this has been discussed before about WebSQL. > The [WebSQL] specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path. https://www.w3.org/TR/webdatabase/

The Chrome blog post about deprecating sqlite-based WebSQL makes an interesing point. I believe it applies to OpenDocument as well.

> The Web SQL specification cannot be implemented sustainably, which limits innovation and new functionality. The last version of the standard literally states "User agents must implement the SQL dialect supported by Sqlite 3.6.19". SQLite was not initially designed to run malicious SQL statements, yet implementing Web SQL means browsers have to do exactly this. The need to keep up with security and stability fixes dictates updating SQLite in Chromium. This comes in direct conflict with Web SQL's requirement of behaving exactly as SQLite 3.6.19.

https://developer.chrome.com/blog/deprecating-web-sql/

Re: What if OpenDocument used SQLite? (2014)

#252

Post by cornstalks »

Eduard wrote: post #244
cornstalks wrote: post #197

Earlier quoted context omitted.

That's what Opus did. The RFC[1] has a base-64 encoded libopus.tar.gz appendix (Appendix A), which is the "primary normative part of this [Opus] specification." If the prose and source code disagree, the source code takes priority and "wins" when it comes to which is normative. I have a love-hate relationship with this approach. [1]: https://datatracker.ietf.org/doc/html/rfc6716

funny, the RFC even includes a shell command pipeline to extract the base64 out of the awkward RFC formatting. Using the C source code still leaves room for ambiguities / under-specification, no? After all, the semantics rely on the particular gcc release used for compiling the code.

There is still the possibility of a bug or under-specification, but that's always the case in any spec. At least with Opus they document what implementation-defined behavior they require, so assuming there aren't any hidden bugs then you should get consistent output across compilers.

Re: What if OpenDocument used SQLite? (2014)

#253

Post by regularfry »

justinclift wrote: post #172
regularfry wrote: post #54

Earlier quoted context omitted.

It also sounds like something that could be manually prevented ahead of time. If you can crack open the file on first save and add the right uniqueness constraint, that should make Audacity crash when it tries to corrupt the data.

> ... that should make Audacity crash when it tries to corrupt the data. That'd be fairly non-optimal behaviour. ;) When the application tries to add wrong data (eg duplicate key violating uniqueness constraint), SQLite will return an error. The application should handle things better than by crashing. In theory anyway. :)

If your choices are a crash or corruption, choose the crash.

Re: What if OpenDocument used SQLite? (2014)

#254

Post by mixmastamyk »

julesnp wrote: post #122
mort96 wrote: post #80

Earlier quoted context omitted.

That feels like it requires the data model to be very different? The file format would essentially need to be a list of changes, with a "committed" flag. Like, if someone changes some text in a paragraph, you can't just model that as "this paragraph now contains this new text". You have to model it as "this paragraph used to contain this text, but an uncommitted change changed it to this other text". User deletes an…

You wouldn't necessarily need to track every change, you could just have 2 tables, one which contains the last "saved" version of the document, and one which contains the last modified version of the document. Upon opening after a crash, if there is a more recent modified version, the program will ask if you want to load that version.

Databases handle all this natively with transactions and WALs. i.e. Don't need to build a Flintstones version yourself.

Also binary documents are a lousy fit with git, smashing square peg into round hole makes little sense.

Re: What if OpenDocument used SQLite? (2014)

#255

Post by mixmastamyk »

dsego wrote: post #249
olddustytrail wrote: post #159

Earlier quoted context omitted.

Libre office does keep a temporary version that allows you to recover, so you're talking crap.

It might very well keep it, but either that behavior wasn't turned on by default or it crashed in way where it wasn't recoverable. I know I lost work.

Doesn't need to be turned on. However it's always possible you mistakenly pressed Esc/Close, didn't read the dialog, or hit a very obscure bug?

However this has worked well for twenty years, so PEBKAC is a reasonable conclusion.

Re: What if OpenDocument used SQLite? (2014)

#256

Post by mixmastamyk »

josephg wrote: post #79
SanderNL wrote: post #75

Earlier quoted context omitted.

You are right and like you explained this is trivially easily fixed by autosaving regularly. What I have trouble imagining is people working with documents on computers for more than a few years yet somehow failing to develop the Always Save Instinct. I regularly catch myself saving unreasonably often.

Users shouldn’t ever need to adapt to computer crashes like this. Software should always auto save or have recovery files or something. As a principle, software should hold anything a user inputs with reverence.

Yes, however you have to be sure every single program across decades is written with those rules in mind. Hard on any OS, but a lost cause on */Linux.

Re: What if OpenDocument used SQLite? (2014)

#257

Post by subleq »

p4bl0 wrote: post #242
remram wrote: post #76

Earlier quoted context omitted.

Why not use a transaction?

A single transaction for the whole user session? That seems a bad idea. Also I'm not sure you can do transactions during another transaction, and I need them for other purpose, i.e., for what they were designed to do (doing changes in multiple tables that need to stay consistent).

It’s exactly what transactions are for. A nested transaction is called a savepoint, which sqlite does support.

Re: What if OpenDocument used SQLite? (2014)

#258

Post by hot_gril »

miki123211 wrote: post #136

The problem with SQLite is that it's not a standardized file format. It's well-documented and pretty well understood for sure, but there's no ISO standard defining how to interpret an SQLite file in excruciating detail. Same goes for competing implementations, Zip and XML have a much smaller API surface than SQLite, whose API, apart from a bunch of C functions, is the SQL language itself. Writing an XML parser is not…

I've never seen this as a problem, since plenty of random things are distributed as sqlite files. All the remaining questions for ODF would be about the schema design.

Re: What if OpenDocument used SQLite? (2014)

#259

Post by hot_gril »

orra wrote: post #141
coliveira wrote: post #139

Earlier quoted context omitted.

But you don't need a standard, because all interaction between applications and the document is made through SQL. And SQL is standardized (at least the parts that matter). If you have concerns about compatibility, make sure that the document can also be accessed through other databases (like mysql).

But other databases cannot access sqlite databases, because the file format is internal...

They can if they want to, using the standard SQLite lib or their own implementation.

Re: What if OpenDocument used SQLite? (2014)

#260

Post by infogulch »

p4bl0 wrote: post #150
rakoo wrote: post #86

Earlier quoted context omitted.

Why do you use a secondary, volatile database ? Performance-wise you won't gain a lot more (we're talking about a user editing a file, so not even 1 write per second). A proposal: write directly, and automatically in the database. No more Save button. There are multiple advantages: - the system is crash-resistant. I like taking the approach of CouchDB where the only correct way to close the system is to crash it. Tha…

> Why do you use a secondary, volatile database ? For the exact reason I gave in the comment you are replying to: I want to keep a usual workflow for users. Principle of least surprise. Users are okay with change being autosaved when there is a single "thing" that can be edited to the point that you don't even have to open it, it's just there, it can be seen as a property (as in ownership) of the application more tha…

Another option is to explicitly start in read-only mode (modification buttons hidden / grayed out, some distinct mode indicator "Viewing Document" next to a button to "Start Editing", etc), and when the user chooses then switch into autosave mode. At this point many users are used to autosave and don't pay due attention to the document saved state. With Microsoft Windows' habit of rebooting your system overnight without your explicit permission, I'm concerned that this might lead to a lot of lost work.
Post reply on HN