Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if OpenDocument used SQLite? (2014)

sqlite.org

Post reply on HN
Print view View original

241–250 of 310 posts

Re: What if OpenDocument used SQLite? (2014)

#241

Post by p4bl0 »

rakoo wrote: post #167
p4bl0 wrote: post #150

Earlier quoted context omitted.

> Why do you use a secondary, volatile database ? For the exact reason I gave in the comment you are replying to: I want to keep a usual workflow for users. Principle of least surprise. Users are okay with change being autosaved when there is a single "thing" that can be edited to the point that you don't even have to open it, it's just there, it can be seen as a property (as in ownership) of the application more tha…

You can keep a distinction between old and new version inside the same database, by having a pointer to the "current" version, and updating the pointer when clicking on "save". You could store all changes in the database in a "staging area", such that when you reopen the app you can load the changes and you don't need a recovery phase, but with the "save" button active meaning that something changed since last save.

I could, but as you said in your previous comment (emphase is mine):

> Less code, fewer bugs.

Re: What if OpenDocument used SQLite? (2014)

#242

Post by p4bl0 »

remram wrote: post #76
p4bl0 wrote: post #6

I'm currently working on an application where I use SQLite as the file format. I want to keep a usual workflow for users where you can make edit to your document and it only changes the file when you save it. So to open a file I copy it into the :memory: database [1], then the user can do whatever manipulation they want and I can directly make the change in the database I don't need to have a model of the document ot…

Why not use a transaction?

A single transaction for the whole user session? That seems a bad idea. Also I'm not sure you can do transactions during another transaction, and I need them for other purpose, i.e., for what they were designed to do (doing changes in multiple tables that need to stay consistent).

Re: What if OpenDocument used SQLite? (2014)

#243

Post by indymike »

amiga386 wrote: post #87
orf wrote: post #14

Coupling a file format to SQLite smells wrong. SQLite is good, but it is also fairly unique in this space. Why? Because it’s hard to replicate everything it does, because it does a lot. But… for this case, do we need it do a lot? No, not really. We don’t need the full SQL standard, a query optimiser, etc etc for basic (+ safe) transaction semantics and the ability to store data in a basic table structure. Perhaps the…

Exactly. Some formats are designed, first and foremost, for interchange. SQLite is pitching that you, as an "app" owner, force the SQLite format upon your users to make it a de-facto standard, without putting the work in to make it a de-jure standard. Show me a formalised ISO / IEC / ANSI / ETSI SQLite standard that the Richard Hipp and his company never deviates from, and the full legal search to ensure there are no…

> SQLite is pitching that you, as an "app" owner, force the SQLite format upon your users to make it a de-facto standard, without putting the work in to make it a de-jure standard.

From TFA:

Note that this is only a thought experiment. We are not suggesting that OpenDocument be changed.

Re: What if OpenDocument used SQLite? (2014)

#244

Post by Eduard »

cornstalks wrote: post #197
joshspankit wrote: post #162

Earlier quoted context omitted.

Sounds like a solution is to use the C implementation to define the standard and have it canonized in to an ISO.

That's what Opus did. The RFC[1] has a base-64 encoded libopus.tar.gz appendix (Appendix A), which is the "primary normative part of this [Opus] specification." If the prose and source code disagree, the source code takes priority and "wins" when it comes to which is normative. I have a love-hate relationship with this approach. [1]: https://datatracker.ietf.org/doc/html/rfc6716

funny, the RFC even includes a shell command pipeline to extract the base64 out of the awkward RFC formatting.

Using the C source code still leaves room for ambiguities / under-specification, no? After all, the semantics rely on the particular gcc release used for compiling the code.

Re: What if OpenDocument used SQLite? (2014)

#245

Post by mcpackieh »

internetter wrote: post #126
Lockal wrote: post #113

Sadly they did not include bad sides: 1) Vulnerabilities: not only in SQLite, but also in wrappers like https://nvd.nist.gov/vuln/detail/CVE-2023-32697 2) Lack of transparency: zip with xml's contains only xml's; meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks. Attempts to fix these issues removes benefits that were mentioned. 3) Lack of implementer support. It was o…

> Vulnerabilities: not only in SQLite, but also in wrappers like Yes, parsing encoded files tends to introduce vulnerabilities. ZIP parsers have had plenty of vulnerabilities. This is not exclusive to SQLite. > Lack of transparency: zip with xml's contains only xml's Both zips and sqlite cannot be read with a text editor. Both are open formats with widely available tools to read them. The sqlite binary might, in fact…

> > meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks.

> Elaborate?

When you delete something from a SQLite database, it isn't necessarily actually removed from the file unless you VACUUM or have the secure_delete PRAMGA turned on. Either of these should solve the problem.

VACUUM INTO is a good way to export sqlite databases from an application for this reason.

Re: What if OpenDocument used SQLite? (2014)

#246

Post by sdeframond »

bane wrote: post #157
miki123211 wrote: post #136

The problem with SQLite is that it's not a standardized file format. It's well-documented and pretty well understood for sure, but there's no ISO standard defining how to interpret an SQLite file in excruciating detail. Same goes for competing implementations, Zip and XML have a much smaller API surface than SQLite, whose API, apart from a bunch of C functions, is the SQL language itself. Writing an XML parser is not…

I'm not sure if the problem you are pointing out has to do with: a) SQLite the file format - which is Public Domain and so well documented that parsers for it exist in numerous other languages even though it's almost pointless because... b) SQLite, the Public Domain (and thus entirely source available) C implementation of the library that can operate on the file format -- and is documented to a level well above what…

I think this has been discussed before about WebSQL.

> The [WebSQL] specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path.

https://www.w3.org/TR/webdatabase/

Re: What if OpenDocument used SQLite? (2014)

#248

Post by galangalalgol »

justin66 wrote: post #206
nelgaard wrote: post #200

Earlier quoted context omitted.

But those are completely different uses of a storage format. Library of congress considers if someone a 100 years from now could write a new importer in whatever langauge/AI they might use by then. Office documents are something you send in email attachments to people you often barely know, and expect them to read it in whatever office system they have. And if the recipient uses e.g., Microsoft Word, OFD/Sqlite might…

It is true that it requires effort for the developers of a software program to support a given file format. Beyond that I'm not sure what your point is.

Not the op, but one point would be, why did we even pick xml, when we had latex and html? Why is a relational database the right tool for a document format?

Re: What if OpenDocument used SQLite? (2014)

#249

Post by dsego »

olddustytrail wrote: post #159
dsego wrote: post #92

Earlier quoted context omitted.

I actually tried using open/libre docs a few years ago just because of it being open source. I was trying to make a point of using locally installed software and avoid google products. Then the thing crashed and I lost an hour of work because it didn't save a temporary version. That's when I gave up on it for good.

Libre office does keep a temporary version that allows you to recover, so you're talking crap.

It might very well keep it, but either that behavior wasn't turned on by default or it crashed in way where it wasn't recoverable. I know I lost work.

Re: What if OpenDocument used SQLite? (2014)

#250

Post by Freak_NL »

dfox wrote: post #203
Freak_NL wrote: post #58

Good article. Although one thing I do like about OpenDocument being just a bunch of XML files in a ZIP archive is that it is fairly easy to generate documents like spreadsheets without using a (potentially hefty) library which knows about the document format. I have a use case where users of a web service want to use data exported as a bunch of rows in a table in a variety of tools. Now, CSV with UTF-8 encoding is of…

As for [1], it is not really hardcoded, but depends on what is the value of localeconv()->decimal_point, if it is “,”, excel uses semicolons both in CSV files and formula expression language. This used to be configurable when opening CSV/TXT file in excel (and still is in LibreOffice) but as a part of the overall UI dumbification was moved somewhere under the “Data” menu/ribbon tab (so you have to open new workbook a…

> decimal_point

Are you sure that affects it? The decimal point parameter sounds like it decides how to write out 5½ (i.e., 5.5 (English style) or 5,5 (Dutch style)) surely? Although on the topic of this particular bête noire I would not be surprised.

Post reply on HN