51–60 of 61 posts
> Alternative, proven Document Databases only offered client/server support. We currently use Realm for this use case. It’s a local database just like SQLite, but with native support for documents and listeners. Did you try that out?
Earlier quoted context omitted.
If you use the library on a server in a node.js environment, wouldn't it be useful to fetch data (e.g. Remix / NextJS)? Besides, I'm not sure if better-sqlite3 offers the listener functionalities I care about. Skimming the docs, it seems it doesn't.
Listener functionality could be something you'd have to write yourself, I suppose. As for on the server, no. Sqlite is a c library, not a separate application- the work happens inside the node process. Regardless of how you do it, any call into sqlite is going to block. Adding promises or callbacks on top of that is just wasting CPU cycles, unlike reading from the filesystem or making a network request, where the wor…
If you are serious about transactionality, data consistency, and isolation levels, this sounds different from how you want to go. Both Firestore and Realm have shortcomings here. Fauna (where I work at, btw) and Surreal with FoundationDB on the backend, and Spanner are the only ones that can guarantee strict serializability in a distributed environment. I could argue that Fauna is the most turnkey (least pain to try,…
If you are serious about transactionality, data consistency, and isolation levels, this sounds different from how you want to go. Both Firestore and Realm have shortcomings here. Fauna (where I work at, btw) and Surreal with FoundationDB on the backend, and Spanner are the only ones that can guarantee strict serializability in a distributed environment. I could argue that Fauna is the most turnkey (least pain to try,…
Addressing main points like implementing atomic transactions (read and write operation on a doc) seems warranted since it exists in Firebase as well.
I'm the core maintainer of the npm sqlite package that your library uses. I recommend that you don't make it a direct dependency, but use dependency injection or an adapter pattern that way your library isn't dependent on a specific package version of sqlite/sqlite3 (the npm sqlite package API is pretty static at this point though!). The npm sqlite package used to have sqlite3 as a direct dependency in older major ve…
So exporting one Database that allows people to pick which driver they want to use might be a simple and user-friendly solution (incl. better-sqlite3) which people have asked for.
I'm the core maintainer of the npm sqlite package that your library uses. I recommend that you don't make it a direct dependency, but use dependency injection or an adapter pattern that way your library isn't dependent on a specific package version of sqlite/sqlite3 (the npm sqlite package API is pretty static at this point though!). The npm sqlite package used to have sqlite3 as a direct dependency in older major ve…
Hey! Thanks for the feedback. I'd just be worried people might not be using the correct libraries since I think they still provide different functionality. So exporting one Database that allows people to pick which driver they want to use might be a simple and user-friendly solution (incl. better-sqlite3) which people have asked for.
- https://github.com/haxtra/kvstore-sqlite (Basic key-value store for SQLite databases.)
- https://github.com/haxtra/super-sqlite3 (Fast SQLite library with optional full db encryption, simple query builder, and a host of utility features, all in one neat package.)
- https://github.com/haxtra/live-object (Standard javascript object with built-in JSON serialization to file. Dreams do come true sometimes.)
All from github user: https://github.com/haxtra
I think the super-sqlite3 source might also be an inspiration for the 'driver' topic: "super-sqlite3 is a thin wrapper around better-sqlite3-multiple-ciphers, which extends better-sqlite3 (the fastest SQLite library for node.js) with full database encryption using SQLite3MultipleCiphers. super-sqlite3 then adds its own query builder and other convenience features."
And do check out this user's XRay (JavaScript object browser component) library for your preferred component framework.
In my bookmarks I also found these other related and interesting links: - https://dgl.cx/2020/06/sqlite-json-support (An article about SQLite as a document database, using the relatively new 'genrated columns' feature of sqlite 3.31.0, which you seem to be using)
- https://www.npmjs.com/package/best.db (easy and quick storage)
- https://tinybase.org (This project seems to be an even more similar idea to Doculite) https://github.com/tinyplex/tinybase (The reactive data store for local-first apps.)
Good luck with your project!
- https://github.com/Rolands-Laucis/Socio
- https://www.youtube.com/watch?v=5MxAg-h38VA&list=PLuzV40bvrS... (video updates of the code and functionality)
Honestly, that sounds absolutely frightening to my ear. There are redis, there is mongo, orient, and plenty of document-based rapid-development databases which will be a substantially better solution than turning sqlite into Firebase.
Yes, you can use data change notification callbacks. It is a cool feature of SQLite, but did you know that their performance is a big concern in a large-scale database (document database grows very quickly by design)? What about COUNTs? Batching operations? Deadlocks? This will go out of hand quickly, because a hammer is used as a shovel here.
>DocuLite lets you use SQLite like Firebase Firestore. Honestly, that sounds absolutely frightening to my ear. There are redis, there is mongo, orient, and plenty of document-based rapid-development databases which will be a substantially better solution than turning sqlite into Firebase. Yes, you can use data change notification callbacks. It is a cool feature of SQLite, but did you know that their performance is a…
>DocuLite lets you use SQLite like Firebase Firestore. Honestly, that sounds absolutely frightening to my ear. There are redis, there is mongo, orient, and plenty of document-based rapid-development databases which will be a substantially better solution than turning sqlite into Firebase. Yes, you can use data change notification callbacks. It is a cool feature of SQLite, but did you know that their performance is a…
I just did a quick search and struggled to find anything about the performance issues you referred to - can you link something so I can take another look? Thanks for the suggestions.
You will unveil more problems with SQLite for this job as you dig deeper. What you really want is fast writes and dumb querying by document id, whereas SQLite gives you an ultra querying suite that you don't utilize but still have to pay with slower writes. This is a classic system-design problem. Just try rewriting your collection and document abstractions using proper document data storage and you will see how less complicated it is.