Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Doculite – Use SQLite as a Document Database

npmjs.com

Post reply on HN
Print view View original

21–30 of 61 posts

Re: Show HN: Doculite – Use SQLite as a Document Database

#21

Post by thenorthbay »

bastawhiz wrote: post #16
thenorthbay wrote: post #8

Earlier quoted context omitted.

Interesting. Updating values via incrementing them is a use case I barely had in Firebase. I mostly only dealt with 1-time updates to values, e.g. by the user or scheduled jobs. In which scenario would the current design cause you problems?

Incrementing is only one possible use case. Any time you read data and then write back based on that data, you need to ensure that nobody wrote to the document in the interim. RDBMS do this with transactions. Consider the case where a user is submitting an e-commerce order. You want to mark their order as processed and submit it for fulfillment. If you read the order to check if it's already submitted, two requests t…

Thank you for the feedback. Seems this is important.

Re: Show HN: Doculite – Use SQLite as a Document Database

#23

Post by jitl »

thenorthbay wrote: post #17
zdragnar wrote: post #14

Why do you have async reads and writes? There's no client-server setup here, using async / await just introduces pointless waiting. https://github.com/WiseLibs/better-sqlite3

If you use the library on a server in a node.js environment, wouldn't it be useful to fetch data (e.g. Remix / NextJS)? Besides, I'm not sure if better-sqlite3 offers the listener functionalities I care about. Skimming the docs, it seems it doesn't.

better-sqlite3 is orders of magnitude faster than the async SQLite bindings. We found this to be true when testing SQLite options for Notion's desktop app anyways. The "why should I use this" bits sound boastful but are reasonable.

https://github.com/WiseLibs/better-sqlite3#why-should-i-use-...

Re: Show HN: Doculite – Use SQLite as a Document Database

#24

Post by zdragnar »

thenorthbay wrote: post #17
zdragnar wrote: post #14

Why do you have async reads and writes? There's no client-server setup here, using async / await just introduces pointless waiting. https://github.com/WiseLibs/better-sqlite3

If you use the library on a server in a node.js environment, wouldn't it be useful to fetch data (e.g. Remix / NextJS)? Besides, I'm not sure if better-sqlite3 offers the listener functionalities I care about. Skimming the docs, it seems it doesn't.

Listener functionality could be something you'd have to write yourself, I suppose.

As for on the server, no. Sqlite is a c library, not a separate application- the work happens inside the node process. Regardless of how you do it, any call into sqlite is going to block. Adding promises or callbacks on top of that is just wasting CPU cycles, unlike reading from the filesystem or making a network request, where the work is offloaded to a process outside of node (and hence why it makes sense to let node do other things instead of waiting).

In fact, if you synchronously read and write within a single function with no awaits or timeouts in-between, you don't have to worry about atomicity- no other request is being handled in the meantime.

Re: Show HN: Doculite – Use SQLite as a Document Database

#25

Post by schemescape »

bastawhiz wrote: post #3

The one feature that I'd want out of this is atomic writes. If I have a document and want to increment the value of a field in it by one, I'm not sure that's possible with Doculite today: if two requests read the same document at the same time and both write an incremented value, the value is incremented by one, not two. The way _I_ would expect to do this is something like this: const ref = db.collection('page').doc…

What about FieldValue.increment()?

https://cloud.google.com/firestore/docs/samples/firestore-da...

Re: Show HN: Doculite – Use SQLite as a Document Database

#26

Post by jitl »

simonw wrote: post #9
thenorthbay wrote: post #5

Earlier quoted context omitted.

Yes – I'm using JSON_extract and generated virtual columns https://www.sqlite.org/json1.html#jex Edit: the database is stored in a sqlite.db file in the cwd

Found where you're using those: https://github.com/thenorthbay/doculite/blob/c05d98c209d0031... It looks like your tables have a single value column and a id generated column that extracts $.id from that value: CREATE TABLE IF NOT EXISTS ${collection} ( value TEXT, id TEXT GENERATED ALWAYS AS (json_extract(value, "$.id")) VIRTUAL NOT NULL ) GENERATED ALWAYS AS was added in a relatively recent SQLite version - 2020-01…

Often sqlite libraries just bundle SQLite instead of relying on the system one, better-sqlite3 does that just fine and has provisions for building against a custom SQLite version if you really need it.

Re: Show HN: Doculite – Use SQLite as a Document Database

#27

Post by jitl »

__jonas wrote: post #12
cyanydeez wrote: post #6

I'd see if you can easily port the on top of browser based sqlite in wasm, that's expand your user base and lead to some of the "holy Grail" in the offline first/sync systems

Why not just use IndexedDB in the browser if you don’t want an SQL database?

a) IndexedDB's API is horrid, so everyone wants to layer an abstraction on top anyways.

b) You can't use IndexedDB on the server, so you wouldn't be able to write sync code that runs on both the client and the server

Re: Show HN: Doculite – Use SQLite as a Document Database

#28

Post by schemescape »

zdragnar wrote: post #14

Why do you have async reads and writes? There's no client-server setup here, using async / await just introduces pointless waiting. https://github.com/WiseLibs/better-sqlite3

I assumed they were implementing the Firestore API (which I’m fairly certain is async).

Re: Show HN: Doculite – Use SQLite as a Document Database

#29

Post by thenorthbay »

mixmastamyk wrote: post #22

Does this mean documents in a database, or a database as a document? I've tried the second, but the time comes when you need to (re)order items which gets clumsy.

If I understood you correctly, Documents in a database, and database as a file. If else please let me know.
Post reply on HN