Live data from Hacker News

SQLite as a Document Database (2020)

dgl.cx

31–40 of 63 posts

Re: SQLite as a Document Database (2020)

#31

Is this not similar to what Simon Willison has written about before? https://sqlite-tutorial-pycon-2023.readthedocs.io/en/latest/... https://simonwillison.net/2021/Jul/28/baked-data/

Check the dates, the "before" part is inaccurate.

I'm sorry, I didn't mean before 'this'. I shouldn't have even said "before". That was me thinking to myself out loud.

What I'm trying to figure out is if they're related concepts. This may be a very naive question awkwardly asked.

Re: SQLite as a Document Database (2020)

#33
post #21
post #20

obviously the example is contrived, but it seems strange me that they are not storing the json in its own column - just extracting a single key from it and storing in a generated column. Why not do that in the app code if youre just going to discard the rest of the json? Here's an example i saw yesterday from mariadb, which is improving its json support in its upcoming releases. https://mariadb.com/docs/server/ha-and…

Postgres does even better and it's available right now. No virtual column needed. CREATE TABLE t1 (data JSONB); INSERT INTO t1 VALUES ('{"column1":1234}'); CREATE INDEX t1column1 ON t1(data->'column1'); SELECT * FROM t1 WHERE data1->'column1' = '1234'; // not sure about data type

I would not call an index format which gets slower with growth "better". And that vacuum issue on JSONB scales with your data size.

Re: SQLite as a Document Database (2020)

#34

Why do people say document database when they really just mean json database?

The 1st edition CouchDB book from 2010 explained it like this: > We write software to improve our lives and the lives of others. Usually this involves taking some mundane information—such as contacts, invoices, or receipts—and manipulating it using a computer application. CouchDB is a great fit for common applications like this because it embraces the natural idea of evolving, self-contained documents as the very cor…

The thing that people always miss about this takeaway is that while it is a truism, most data is actually inherently relational. Even in your example given, the individual components that are made to assemble that document are better represented as relational datastores

Re: SQLite as a Document Database (2020)

#36

Earlier quoted context omitted.

The 1st edition CouchDB book from 2010 explained it like this: > We write software to improve our lives and the lives of others. Usually this involves taking some mundane information—such as contacts, invoices, or receipts—and manipulating it using a computer application. CouchDB is a great fit for common applications like this because it embraces the natural idea of evolving, self-contained documents as the very cor…

The thing that people always miss about this takeaway is that while it is a truism, most data is actually inherently relational. Even in your example given, the individual components that are made to assemble that document are better represented as relational datastores

Nothing here says the data isn't relational. It strongly disagrees, with reasons, why it's not better represented as relational.

Personally I prefer the relational stance, and there are a lot of people who don't get it who say things like "this data isn't relational", but that's not the argument GP made.

Re: SQLite as a Document Database (2020)

#38
Since sqlite people are probably in this thread - why is storing genomic data as a sqlite tar with all the metadata you want in tables a “Bad Idea” (tm)? Like toss a fastq or bam plus all the downstream data, grant info, experiment parameters, specimen info etc etc in a single file easily parsable.

Re: SQLite as a Document Database (2020)

#39

Why do people say document database when they really just mean json database?

The "document" is sort of the native data type, Put everything in a big hierarchical structure, It is very flexible but analytics across the set can suffer. "relational" is another way to store data, break your big hierarchy into sets of related rows and store the rows as a table, If I were to describe it in geometric terms where the document is a report on a paper, the relation is a vertical slice through a stack of those reports. This is slightly non-intuitive but provides for interesting analysis opportunities.

But nothing prevents you from treating your relational database as a document database, set it up as a key-value store where each key is is the document title and each value is a large blob of document data. If your documents are fairly consistent it is also easy enough to build indexes and query features to regain some of the analytical ability of relational data.

Re: SQLite as a Document Database (2020)

#40
I developed an application using zope on the backend and extjs (now Sencha) and SQLite on the front end in 2009. The form + data was stored as strings in the database and then synced to back-end when connected to the internet. It was targeting remote doctors in third world countries who were often offline. Several doctors at that time had told me they wanted to store the medical data in the same format as the intake form mostly because that was what they were used to with paper forms. Soon after, there was a big push for medical ERP and relational databases won over document databases. I bet it would be much easier to build an application like that now (data stored and displayed in same format as collected) but wonder what market would use it.
Post reply on HN