D1: Our SQL database

This is really interesting, it's (basing it on SQLite) exactly what I was expecting CloudFlare to do for their first DB.

Its perfect for content type sites that want search and querying.

Anyone from CF here, is it using Litestream (https://litestream.io) for its replication or have you built your own replication system?

I assume this first version is somewhat limited on write performance having a single "main" instance and SQLite laking concurrent writes? It seems to me that using SQLite sessions[0] would be a good way to build an eventually consistent replication system for SQLite, would be perfect for an edge first sql database, maybe D2?

0: https://www.sqlite.org/sessionintro.html

Now is this a Cloudflare ($NET) buy signal? I think you know the answer.

Maybe they will announce a Hashicorp competitor in their next reveal. Who knows.

wow SQLite getting a lot of love these days

https://tailscale.com/blog/database-for-2022

https://fly.io/blog/all-in-on-sqlite-litestream

https://blog.cloudflare.com/introducing-d1

At any sufficient scale, it is difficult to do filtering on the client. Yes, it can be done, but with 10,000+ potential records, you don’t want to ship that to the client for each query. (Note: I’m thinking Newegg scale for “hdmi cable” here. There are certainly situations where you can ship the entire database to the client for filtering.)

It’s not DOM bloat… it’s too many records. If you’re building a DOM node for each record, that’s bloat, but you still have the problem even if the results are stored in a JSON object and dynamically queried on the client side.

So, for each new filter or new query you need to hit the server anyway. If that’s an asynchronous query that returns a json blob or a full refresh, IMHO, it doesn’t really matter that much. Either way, you’re rebuilding a large portion of the DOM with the new results. The only thing that skews things in favor of an async call is if the rest of the page is so heavyweight that reloading the page takes a significant amount of time. This is probably what you’re taking about.

Having a SQLite db close to your worker node really isn’t going to affect this problem all that much.

We are definitely interested in ORMs. Want to make it easy to use. I hope someone creates the next Rails using Workers. And having other models on top of our SQL offerings will be important. Get in contact and let us know what you'd like.