41–50 of 165 posts
> We’re actively working on global read replication and realizing the above proposal (share feedback In the #d1 channel on our Developer Discord).
Perhaps it will be out by the time the book is finished.
[1] https://blog.cloudflare.com/building-d1-a-global-database
What portion of writes typically have to go to the primary server? If users are local then their data maybe is often local as well. In other words not all writes need to first go the central server from where they are distributed back to all edge-databases. I can see that the organization running the application needs a global view of all data. But regional users perhaps don't. Often they just need to know their own…
Say you flag a write as "local". Later, some other place in your app starts relying on this write in another locality. If you don't update your write spec from "local" to "primary", you don't have a consistent database anymore, but you will make decisions thinking that you do.
Now consider a team of 10. Or 20...
Mayhem can spiral very quickly from there.
Rather than a paradigm shift or hype, I see distributed SQLite as an extension of a path that devs can go down. With Litestream, the most common complaint I got was that devs were worried that they couldn't horizontally scale with SQLite and they'd be stuck. While you probably won't hit vertical scaling limits of SQLite on most projects, it still caused concern. So LiteFS became a "next step" that a dev could take if they ever got to that point. It doesn't need to be your starting point.
As for the "hacky" solution of txid, I'm not sure why that's hacky. Your application isn't required to use it or the optional built-in proxy but it's available if it fits your application's needs. It also works for plugging legacy applications into distributed SQLite without retrofitting the code. The proposed solution of caching seems orthogonal to the discussion of distributed application data. I don't think any database provider would suggest to avoid caching when it's appropriate but there's plenty of downsides of caching. Hell, it's one of the two hardest problems in computer science.
I don't believe Cloudflare D1 has read replicas yet? They describe how it will work in detail in a blog post [1], but in the future tense. > We’re actively working on global read replication and realizing the above proposal (share feedback In the #d1 channel on our Developer Discord). Perhaps it will be out by the time the book is finished. [1] https://blog.cloudflare.com/building-d1-a-global-database
Last year's major Cloudflare outage really was the final straw for me with regards to D1. I don't mean that as a knock at Cloudflare at all, the situation sounded horrible and I appreciate how the entire team responded to it. I just worry that internal responses to fundamental infrastructure issues will leave newer projects like D1 on ice for a year or two.
SQLite has pros and cons, like anything else. When it comes to web frameworks and app platforms, to fundamental problem is that many committed fully to server less/edge and ignored the los of persistent storage. You just can't use a local database when using serverless, or any type of distributed compute/rendering for that matter.
Databases are centralized by design, you can dodge some of that complexity with clever synchronization protocols but you are still limited to having a single primary DB at the end of the day.
For read-heavy use cases, tools like Turso can be invaluable. If database writes are more common, you'll always be limited by network latency. More importantly for most modern web apps, whether you render HTML in the browser or a server you can't avoid loading states. IMO you might as well lean on the platform and use server rendering whenever persistent state is involved.
I wonder if SQLite will ever be modified to allow concurrent writes. Right now it locks the whole DB on write [1]. But since SQLite usage is rapidly expanding from its embedded db roots, one has to wonder if the devs have considered creating some sort of SQLite daemon, or if the underlying architecture would have to be changed so drastically it wouldn't be worth it. 1. https://www.sqlite.org/faq.html#q5
this page just says 'Forbidden'
SQLite is getting a lot heavier. This is becoming more like MySQL or Postgres.
SQLite is getting a lot heavier. This is becoming more like MySQL or Postgres.