Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What If OpenDocument Used SQLite?

sqlite.org

Post reply on HN
Print view View original

41–50 of 109 posts

Re: What If OpenDocument Used SQLite?

#41

Post by thayne »

jll29 wrote: post #31

I love SQLite. As a document _exchange_/_interchange_ format, what I prefer for durability is a non-binary format (e.g. XML based). For local use, I agree SQLite might be much faster than ZIP, and of course the ability to query based on SQL has its own flexibility merits.

XML isn't great for exchange/interchange either due to security problems and inconsistencies in implementations. A big part of the problem is that xml has a lot of complexity, which leads to a bigger attack surface when parsing and processing untrusted data. And then xml entities are just inherently insecure, unless you disable some of their capabilities (like using remote files, and unlimited recursion).

That said, creating a format that can convey rich untrusted data is a hard problem.

Re: What If OpenDocument Used SQLite?

#42

Post by korkor55 »

chmaynard wrote: post #9

Dr. Hipp occasionally gets on a soapbox and extolls the virtue of sqlite databases for use as an application file format. He also preaches about the superiority of Fossil over Git. His arguments generally make sense. I tolerate his sermons because he is one of the truly great software developers of our time, and a personal hero of mine.

These are thought-experiments to help better understand how SQLite works. This is exactly how supporting documentation should be written so that others read it.

He even went over the top with the disclaimers.

Re: What If OpenDocument Used SQLite?

#43

Post by charleslmunger »

agwa wrote: post #28

If you're going to use SQLite as an application file format, you should: 1. Enable the secure_delete pragma https://antonz.org/sqlite-secure-delete/ > so that when your user deletes something, the data is actually erased. Otherwise, when a user shares one of your application's files with someone else, the recipient could recover information that the sender thought they had deleted. 2. Enable the options described at…

>and they seem to get annoyed that people run fuzzers against SQLite, even though application file formats should definitely be fuzzed.

I think that's an unfair reading. Sqlite runs fuzzers itself and quickly addresses bugs found by fuzzers externally. There's an entire section in their documentation about their own fuzzers and thanking third party fuzzers, including credit to individual engineers.

https://www.sqlite.org/testing.html

The tone of the CVE docs are because people freak out about CVEs flagged by automated tools when the CVEs are for issues that have no security impact for typical usage of SQLite, or have prerequisites that would already have resulted in some form of compromise.

Re: What If OpenDocument Used SQLite?

#44

Post by wakawaka28 »

tombert wrote: post #25

I remember I played with some software called "The Illumination Software Creator" [1], and I remember the saved project files were just SQLite databases. I actually thought it was kind of cool, because I was able to play with it easily with some SQLite explorer tool (I forget which one) and I could easily look at how the save files actually worked. I haven't really used SQLite for anything serious [2], but always fou…

What is it that makes you think Lunduke is pseudo-intellectual? He certainly doesn't try to pose as a scholar. If you are like most of his haters, you just refuse to believe that smart people can be conservatives.

Re: What If OpenDocument Used SQLite?

#45

Post by afiori »

lifthrasiir wrote: post #34

SQLite can't be reliably used in networked file systems because it heavily relies on locking to be correctly implemented. I recently had to add a check for such file systems in my application [1] because I noticed a related corruption firsthand. Simpler file formats do not demand such requirements. SQLite is certainly good, but not for this use. [1] https://github.com/lifthrasiir/angel/commit/50a15e703ef2c1af...

In that case the application would keep a temporary file and copy over when saving

Re: What If OpenDocument Used SQLite?

#46

Post by kstrauser »

wakawaka28 wrote: post #44
tombert wrote: post #25

I remember I played with some software called "The Illumination Software Creator" [1], and I remember the saved project files were just SQLite databases. I actually thought it was kind of cool, because I was able to play with it easily with some SQLite explorer tool (I forget which one) and I could easily look at how the save files actually worked. I haven't really used SQLite for anything serious [2], but always fou…

What is it that makes you think Lunduke is pseudo-intellectual? He certainly doesn't try to pose as a scholar. If you are like most of his haters, you just refuse to believe that smart people can be conservatives.

There’s no way to discuss Lunduke without getting into politics, so I’ll leave it that Lunduke is clearly a very intelligent person who IMO mistakes his knowledge in some areas for general expertise in other unrelated fields.

It’s a common trap to fall into. See also: Ben Carson. Both of them are obviously intelligent and highly skilled in their professional fields. And both have let that convince themselves that they know everything about everything.

Re: What If OpenDocument Used SQLite?

#48

Post by scott_w »

floating-io wrote: post #3

An interesting skim, but it would have been more meaningful if it had tackled text documents or spreadsheets to show what additional functionality would be enabled with those beyond "versioning". Maybe it's just me, but I see the presentation functionality as one of the less used aspects of the OpenOffice family.

While reading I was musing one way to handle text could be to use a linked list format as storage? To make it work like that, you’d need the editor to work on a block concept and I don’t think document editors work like that?

Spreadsheets might be a little easier because you can separate out by sheet or even down to a row/column level?

Part of me wants to try it now…

Re: What If OpenDocument Used SQLite?

#49

Post by treyd »

ignoramous wrote: post #27

> SQLite database has a lot of capability, which this essay has only begun to touch upon. But hopefully this quick glimpse has convinced some readers that using an SQL database as an application file format is worth a second look. It really is. One of the experiments we have been doing currently to make bug reporting from Androids easier (and to an extent, reduce user frustration and fatigue) is to store app logs (un…

What kinds of queries are being done on the logs such that it makes sense to use sqlite instead of, like, just a ring buffer?
Post reply on HN