AutoCAD uses a database as its file format, it is fairly slow.
What if OpenDocument used SQLite? (2014)
121–130 of 310 posts
Re: What if OpenDocument used SQLite? (2014)
#122Earlier quoted context omitted.
If you have a db, why not just model it as unsaved data? I.e. all changes get stored to the db, but have a flag of unsaved. If you open up a file and there are unsaved changes, you can prompt the user to either make them saved or discard them.
That feels like it requires the data model to be very different? The file format would essentially need to be a list of changes, with a "committed" flag. Like, if someone changes some text in a paragraph, you can't just model that as "this paragraph now contains this new text". You have to model it as "this paragraph used to contain this text, but an uncommitted change changed it to this other text". User deletes an…
You wouldn't necessarily need to track every change, you could just have 2 tables, one which contains the last "saved" version of the document, and one which contains the last modified version of the document. Upon opening after a crash, if there is a more recent modified version, the program will ask if you want to load that version.
Re: What if OpenDocument used SQLite? (2014)
#123Sadly they did not include bad sides: 1) Vulnerabilities: not only in SQLite, but also in wrappers like https://nvd.nist.gov/vuln/detail/CVE-2023-32697 2) Lack of transparency: zip with xml's contains only xml's; meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks. Attempts to fix these issues removes benefits that were mentioned. 3) Lack of implementer support. It was o…
4) is enough for me, so I agree with your general point, but 1) 2) and 3) aren't really cons for SQLite.
1) Makes sense only if the average XML parsers and zip libraries in use have fewer vulnerabilities and are actively maintained as well.
2) You can store sensitive data in a SQLite database or XML file, there's no real difference. You can clean up a SQLite database pretty easily if you want and that doesn't take away all the benefits.
3) What does implementer support even mean? I believe they are open to custom work... WebSQL died because it doesn't make sense to pretend SQLite is some kind of standard -- that brings us back to 4), which is the valid reason to avoid SQLite.
Actually, your 4) is worded too strongly. They say they're committed to forward compatibility as long as you don't use the new features. That makes forward compatibility the decision of the app: an app can have forward compatibility and not use newer features OR lose forward compatibility and use newer features.
Re: What if OpenDocument used SQLite? (2014)
#124Earlier quoted context omitted.
Why do you use a secondary, volatile database ? Performance-wise you won't gain a lot more (we're talking about a user editing a file, so not even 1 write per second). A proposal: write directly, and automatically in the database. No more Save button. There are multiple advantages: - the system is crash-resistant. I like taking the approach of CouchDB where the only correct way to close the system is to crash it. Tha…
a save button is still good, as it allows you to keep specific checkpoints. but the save button could simply tag specific save points in a larger table. if the format can roll up changes to compress them, they also indicate where which variants need to be kept indefinitely.
In this view the save action is more like a commit, where the user manually checkpoints and also offers a simple description of why this is an important point. But in my view all intermediary points also need to be saved, because the user might have forgotten to explicitly checkpoint, and might still want some undo/redo capability that is more granular than just checkpoints.
Re: What if OpenDocument used SQLite? (2014)
#125Earlier quoted context omitted.
That may have been true years ago in win 95 or xp days. The modern paradigm starting with google docs is that things are automatically saved and even always sharable through the cloud, making manual saving actually an atavistic leftover of a bygone era.
What if I actually don't want any changes saved because I've only opened a document for reference purposes?
Modern apps like gdocs have a toggle for switching between read-only, suggest changes, and editing. If you forgot to toggle, you can just open version history and revert. MS Office also had version history for nearly a decade.
Re: What if OpenDocument used SQLite? (2014)
#126Sadly they did not include bad sides: 1) Vulnerabilities: not only in SQLite, but also in wrappers like https://nvd.nist.gov/vuln/detail/CVE-2023-32697 2) Lack of transparency: zip with xml's contains only xml's; meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks. Attempts to fix these issues removes benefits that were mentioned. 3) Lack of implementer support. It was o…
> Vulnerabilities: not only in SQLite, but also in wrappers like
Yes, parsing encoded files tends to introduce vulnerabilities. ZIP parsers have had plenty of vulnerabilities. This is not exclusive to SQLite.
> Lack of transparency: zip with xml's contains only xml's
Both zips and sqlite cannot be read with a text editor. Both are open formats with widely available tools to read them. The sqlite binary might, in fact, be more widely available than unzipping tools.
> meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks.
Elaborate?
> Lack of implementer support. It was one of the reasons for WebSQL deprecation many years ago.
I don't understand how this is relevant?
> SQLite does not even promise forward compatibility, only backward one. Which means that new documents might not open in old software
Neither does OpenDocument. SQLite is actually more solid in this regard – forwards compatibility is still a thing unless new features are used.
Re: What if OpenDocument used SQLite? (2014)
#127As an aside, this blew me away. I can hardly believe it. No nested query required? > SELECT manifest, versionId, max(checkinTime) FROM version; > "Aside: Yes, that second query above that uses "max(checkinTime)" really does work and really does return a well-defined answer in SQLite. Such a query either returns an undefined answer or generates an error in many other SQL database engines, but in SQLite it does what yo…
It's not really what one would expect in SQL, but SQLite often defies expectation. In this case, handy, but non-standard.
Re: What if OpenDocument used SQLite? (2014)
#128Earlier quoted context omitted.
> Yes it should replicate the functionality user expects Do users really expect this now a days? Most users use cloud apps, and almost all of those save after every operation automatically.
Which is a compromise for using browsers really. It isn't a good solution and no user really understands this and I believe it is the most hated feature of the new cloud world. Yes, leaving the page open for multiple hours might now allow you to save because your access token expired. No, communication in the background is unreliable too. Autosave is a bad band aid for a bad solution. Doing periodic and automatic sav…
Its pretty easy to make a cloud app that emulates the traditional working draft/save workflow. Browsers all have pretty reliable local storage technology now a days if your network is unstable. I don't think this design choice is a compromise of the medium. If anything it seems like if you were going to have to compromise for web you would do it in the other direction so apps are more usable during poor network conditions.
I would say the traditional model is a compromise from back when disks were unacceptably slow to be saving constantly.
Re: What if OpenDocument used SQLite? (2014)
#129Re: What if OpenDocument used SQLite? (2014)
#130Earlier quoted context omitted.
Yes it should replicate the functionality user expects - save everything into temporary file and overwrite the original file only on explicit save action. As for Git, it would benefit from using text format specifically aimed for easy diffing/merging. No idea how easy the sqlite dump is in this regard.
> As for Git, it would benefit from using text format specifically aimed for easy diffing/merging. No idea how easy the sqlite dump is in this regard. The problem I'd predict here is that then people would expect to be able to do three-way merges. It might even work correctly a lot of the time, depending on the exact pattern of changes. But my gut feel is that unless the schema were designed just right, there would b…
If you want to use flat files you should just use flat files. There are plenty of unix tools to treat them like DB.
You're not going to have a sensible text version of a btree that is reasonably editable by a text editor.