Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if OpenDocument used SQLite? (2014)

sqlite.org

Post reply on HN
Print view View original

91–100 of 310 posts

Re: What if OpenDocument used SQLite? (2014)

#91

Post by floppydiscen »

5e92cb50239222b wrote: post #63
ncruces wrote: post #43

Earlier quoted context omitted.

Which is also "not a big issue", since it's a recommended Library of Congress storage format, and supported long term: https://www.sqlite.org/locrsf.html https://www.sqlite.org/lts.html

It is somewhat of a problem: the development team is very small, they don't take outside contributions (so nobody outside the core team really builds up expertise over time), and the vast majority of tests are proprietary. I hope they have a contingency plan just in case (some sort of a dead man's switch that publishes the test suite under a permissible license) as it would probably be quite difficult for others to m…

I'm pretty sure this is why libsql was created https://github.com/libsql/libsql

Re: What if OpenDocument used SQLite? (2014)

#92

Post by dsego »

chrisshroba wrote: post #83
SanderNL wrote: post #75

Earlier quoted context omitted.

You are right and like you explained this is trivially easily fixed by autosaving regularly. What I have trouble imagining is people working with documents on computers for more than a few years yet somehow failing to develop the Always Save Instinct. I regularly catch myself saving unreasonably often.

I used to have that instinct but lost it in the age of auto save. The applications (web or native) I use most often all do it for me: Google docs, Dropbox paper, notion, vscode. I don’t think I’m alone in this!

I actually tried using open/libre docs a few years ago just because of it being open source. I was trying to make a point of using locally installed software and avoid google products. Then the thing crashed and I lost an hour of work because it didn't save a temporary version. That's when I gave up on it for good.

Re: What if OpenDocument used SQLite? (2014)

#93

Post by tuyiown »

bawolff wrote: post #34
tuyiown wrote: post #26

Earlier quoted context omitted.

App using a format specific to their own and unique implementation, that ends up kind of proprietary is perfectly ok. Using it for a open specification which target is cross implementation compatibility makes the move way more hazardous. Meaning, every implementation has to run on environment targetable and compatible wit sqlite or has to re-implement a compatibility layer on something complex enough that you only re…

> Meaning, every implementation has to run on environment targetable and compatible wit sqlite Well i get what you are saying, sqlite has been ported all over the place. It probably wouldn't be the limiting factor portability wise.

Yes, right now, no problem, and there little foreseeable future where a sqlite ported to anything would pose a problem. Still, decisions with no way back like this one requires extra cautions.

Re: What if OpenDocument used SQLite? (2014)

#94

Post by fweimer »

5e92cb50239222b wrote: post #63
ncruces wrote: post #43

Earlier quoted context omitted.

Which is also "not a big issue", since it's a recommended Library of Congress storage format, and supported long term: https://www.sqlite.org/locrsf.html https://www.sqlite.org/lts.html

It is somewhat of a problem: the development team is very small, they don't take outside contributions (so nobody outside the core team really builds up expertise over time), and the vast majority of tests are proprietary. I hope they have a contingency plan just in case (some sort of a dead man's switch that publishes the test suite under a permissible license) as it would probably be quite difficult for others to m…

But that equally applies to getting critical bug fixes for your particular usage scenario of SQLite. It's not just about the viability as a storage format.

For the latter, because the stored data has such a simple format and the implementation has so few dependencies, I expect it will be very easy to get your data out for a long time to come. It's going to be tougher if you have business logic in views or other SQL expressions, of course, and if you rely on SQLite's particular approach to data types (as in “values have types“, but not much more).

Re: What if OpenDocument used SQLite? (2014)

#95

Post by rakoo »

orf wrote: post #14

Coupling a file format to SQLite smells wrong. SQLite is good, but it is also fairly unique in this space. Why? Because it’s hard to replicate everything it does, because it does a lot. But… for this case, do we need it do a lot? No, not really. We don’t need the full SQL standard, a query optimiser, etc etc for basic (+ safe) transaction semantics and the ability to store data in a basic table structure. Perhaps the…

To add to your point, fossil, the versioning system designed by the people of SQLite, and using SQLite, doesn't even use SQLite as a file format. It's all a bunch of blobs, each with its own format, that happen to be stored on SQLite. SQLite offers safe storage and a bunch of helpful indexes and views, but is not necessary for fossil-the-data to work.

Re: What if OpenDocument used SQLite? (2014)

#96

Post by isoprophlex »

sgu999 wrote: post #52
isoprophlex wrote: post #50

Man do I love SQLite. Over the past 1.5 yrs I've build a computer vision tool from recording hardware/software, to derp learning pipelines, to front-end; we had some requirements on the recording side that were difficult to solve with existing solutions (storing exactly timestamped camera frames, gps data, car telemetry and other metadata). Using a SQLite-backed data format for the video recordings made implementing…

I'm working on a similar problem and I've been struggling to convince all my colleagues that we should sqlite most things. By any chance do you have some public code, or blog posts to share?

Not in public repos, but sure. Drop me a line, hn at rombouts dot email.

Re: What if OpenDocument used SQLite? (2014)

#98

Post by Etheryte »

cm2187 wrote: post #85

Sqlite format is smaller than the original format only because xml is super verbose, so any uncompressed binary format ends up being less than lightly zipped xml. But sqlite files aren't small. One thing I don't understand is why they don't do string deduplication in sqlite (as in you only store a string once and every other occurence is just a pointer to that string). It seems such an obvious and easy way to reduce…

My first guess is that if you always store the full string you don't need to scan the database to see if you already have the same string. Essentially you choose to use more space but reduce load. Regardless of whether you do the string deduping on inserts or async later on, you have to do it at some point and the unpredictable performance overhead might be undesirable.

Re: What if OpenDocument used SQLite? (2014)

#100

Post by eviks »

im3w1l wrote: post #49

I don't want people to read my drafts. That could be highly embarassing, and they should not make it into the final saved document. Past version and undo history should be stored separately from the document. They should be stored out of tree where they wont be commited into some git repository or be automatically synced or anything like that.

Then don't give people access to your drafts but exported versions without history? Why put the limits on the efficiency of a format by forcing it to store changes elsewhere?
Post reply on HN