Starbase: SQLite on the Edge • Hacker News Board

Re: Starbase: SQLite on the Edge

#21

Post by pixelatedindex » Mon, Oct 07, 2024, 12:07 AM UTC

I find the website design to be quite charming. Actually evokes some of the Outer World game vibes.

Re: Starbase: SQLite on the Edge

#22

Post by [deleted] » Mon, Oct 07, 2024, 12:34 AM UTC

Diti wrote: post #16

aliasxneo wrote: post #5

It appears the blog links are broken. Are those also supposed to function as the documentation?

Same here. I wonder if it’s only broken for us or if it’s broken for everybody else as well.

[deleted]

Re: Starbase: SQLite on the Edge

#23

Post by brayden_wilmoth » Mon, Oct 07, 2024, 12:36 AM UTC

_hyn3 wrote: post #2

This is pretty cool. Great artwork too. Building edge DBs on top of S3 or R2 had interested me for a long time. Thanks for writing it! Update: bug in redirection: https://starbasedb.com/blog/blog/blog/blog/blog/blog/develop...

Had no idea this was put on HN today. Author of StarbaseDB here, will get a bug fix for the blog redirection pushed out soon! Thanks for bringing this up.

Re: Starbase: SQLite on the Edge

#24

Post by burcs » Mon, Oct 07, 2024, 12:37 AM UTC

ec109685 wrote: post #7

How does this compare with Cloudflare’s new offering SQLite it durable objects: https://blog.cloudflare.com/sqlite-in-durable-objects

So it actually sits on top of it, providing an HTTP interface to interact with them.

Re: Starbase: SQLite on the Edge

#25

Post by brayden_wilmoth » Mon, Oct 07, 2024, 1:02 AM UTC

dantiberian wrote: post #9

I don't really understand what this is offering beyond Cloudflare's recent release of running SQLite in durable objects: https://blog.cloudflare.com/sqlite-in-durable-objects/ . Is it about providing an external interface to Cloudflare's SQLite databases?

Still think there is a lot we can add to StarbaseDB to make the developer experience on SQLite databases better, and personally I think it starts with a better external interface. Provide a frictionless way to get started. Then figuring out how to improve developer experiences on how they interface with databases.

Is it auto-accessible REST endpoints? Easy to implement websocket support for the database? Data replication for scaling reads? Offline data syncing? A lot of potential wins for a layer like this to build on.

Re: Starbase: SQLite on the Edge

#26

Post by burcs » Mon, Oct 07, 2024, 1:02 AM UTC

Diti wrote: post #16

aliasxneo wrote: post #5

It appears the blog links are broken. Are those also supposed to function as the documentation?

Same here. I wonder if it’s only broken for us or if it’s broken for everybody else as well.

Sorry about that, they should be fixed now!

Re: Starbase: SQLite on the Edge

#27

Post by bigiain » Mon, Oct 07, 2024, 1:12 AM UTC

pixelatedindex wrote: post #21

I find the website design to be quite charming. Actually evokes some of the Outer World game vibes.

I found it to be hilariously uninformative.

I have this old school expectation that I should be able to understand what your website/product/project is about but reading the first few sentences in the "above the fold" section of your homepage. I don't want to play a game or click below the fold "read more" links to try and find out what this web content has to do with the domain name "starbasedb".

Perhaps they actually know their intended (seemingly game developer?) audience that I give them credit for? But I do have plans (admittedly only very vague plans right now) to use one of the available "SQLite from javascript in a webpage querying the SQLite file in an object store like S3 without a web backend" techniques or projects. Maybe they'd work for me with my non game oriented use case? I don't know, it seems like they don't want me to find out...

Re: Starbase: SQLite on the Edge

#28

Post by burcs » Mon, Oct 07, 2024, 1:21 AM UTC

rmbyrro wrote: post #3

This is very appealing. If the authors are reading: What is the architecture overview? Does each worker have a copy or do all of them work on the same DB files? How does it handle concurrent workers and db locking? Is SQLite native locking mechanism sufficient in this application?

Multiple workers would connect to the same durable object and write to it, basically, the Durable Object takes care of all the writes in a single thread, so there's no issues with concurrent modifications.

We probably need to build out some sort of data replication to scale reads though Durable Objects are limited to 1000 RPS.

Re: Starbase: SQLite on the Edge

#29

Post by rmbyrro » Mon, Oct 07, 2024, 1:21 AM UTC

From what I could understand of their architecture [1], you may lose data when the machine running DO fails.

DO implements a very comprehensive strategy to mitigate data loss in DO backed by SQLite. But Starbase seems to add a layer of query caching on the application level, which seems to be naively implemented (I might be misunderstanding, please correct if I'm wrong).

[1] https://starbasedb.com/blog/developing-acid-transaction-supp...

Re: Starbase: SQLite on the Edge

#30

Post by brayden_wilmoth » Mon, Oct 07, 2024, 1:50 AM UTC

rmbyrro wrote: post #29

From what I could understand of their architecture [1], you may lose data when the machine running DO fails. DO implements a very comprehensive strategy to mitigate data loss in DO backed by SQLite. But Starbase seems to add a layer of query caching on the application level, which seems to be naively implemented (I might be misunderstanding, please correct if I'm wrong). [1] https://starbasedb.com/blog/developing-aci…

We don't cache query responses in the Starbase application layer. When we originally set out to build this, we were under the assumption we had to handle the ACID transactional support ourselves. Thankfully someone from Cloudflare shed some light for us as you can see in our Github issues and open PR's :)

Learn more about that here: https://github.com/Brayden/starbasedb/issues/12