21–30 of 158 posts
Does anyone else bump into the issue, that the fly.io website does not load if requested via IPv6 on Mac? I tried Safari, Chrome and curl and neither work: $ curl -v https://fly.io/blog/introducing-litefs/ * Trying 2a09:8280:1::a:791:443... * Connected to fly.io (2a09:8280:1::a:791) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem…
> Developing against a relational database requires devs to watch out for "N+1" query patterns, where a query leads to a loop that leads to more queries. N+1 queries against Postgres and MySQL can be lethal to performance. Not so much for SQLite. This is misleading AFAICT. The article(s) is actually comparing remote RDBMS to local RDBMS, not Postgres to SQLite. Postgres can also be served over a UNIX socket, removing…
Well that was fast... [1] Are the readable replicas supposed to be long-lived (as in, I don't know, hours)? Or does consul happily converge even with ephemeral instances coming and going every few minutes (thinking of something like Cloud Run and the like, not sure if Fly works the same way)? And do they need to make a copy of the entire DB when they "boot" or do they stream pages in on demand? [1] https://news.ycomb…
Right now, yes, they should be relatively long-lived (e.g. hours). Each node keeps a full copy of the database so it's recommended to use persistent volumes with it so it doesn't need to re-snapshot on boot.
We do have plans to make this work with very short-lived requests for platforms like Lambda or Vercel. That'll use transactionally-aware paging on-demand. You could even run it in a browser, although I'm not sure why you would want to. :)
ELI5?
It’s a tool to allow you to keep your database on the app server and backed up on s3. This means you don’t need a separate database server. Having a database on the same machine as the app is very fast, but it is usually risky. The backups made by LiteFS lessen the risk.
> Developing against a relational database requires devs to watch out for "N+1" query patterns, where a query leads to a loop that leads to more queries. N+1 queries against Postgres and MySQL can be lethal to performance. Not so much for SQLite. This is misleading AFAICT. The article(s) is actually comparing remote RDBMS to local RDBMS, not Postgres to SQLite. Postgres can also be served over a UNIX socket, removing…
Anyone want to chime in with the largest app they've deployed where prod Postgres was reachable over a Unix domain socket?
No practical way to test this, of course.
Does anyone else bump into the issue, that the fly.io website does not load if requested via IPv6 on Mac? I tried Safari, Chrome and curl and neither work: $ curl -v https://fly.io/blog/introducing-litefs/ * Trying 2a09:8280:1::a:791:443... * Connected to fly.io (2a09:8280:1::a:791) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem…
> Developing against a relational database requires devs to watch out for "N+1" query patterns, where a query leads to a loop that leads to more queries. N+1 queries against Postgres and MySQL can be lethal to performance. Not so much for SQLite. This is misleading AFAICT. The article(s) is actually comparing remote RDBMS to local RDBMS, not Postgres to SQLite. Postgres can also be served over a UNIX socket, removing…
Anyone want to chime in with the largest app they've deployed where prod Postgres was reachable over a Unix domain socket?
> Developing against a relational database requires devs to watch out for "N+1" query patterns, where a query leads to a loop that leads to more queries. N+1 queries against Postgres and MySQL can be lethal to performance. Not so much for SQLite. This is misleading AFAICT. The article(s) is actually comparing remote RDBMS to local RDBMS, not Postgres to SQLite. Postgres can also be served over a UNIX socket, removing…
Isn't "n + 1" typically a degenerate case of ORM rather than a relational thing? The whole point of rdbms is you can do a join instead of iterated lookups.
You definitely don't need an ORM to get to N+1 hell, a for-loop in vanilla PHP will do.
Earlier quoted context omitted.
Note: I apologize if this is overstepping, its hard to tell! I think a strong - extremely strong - selling point is the point I made about "prebaked" data for your APIs, since the entire strength of these SQLite based systems reside in their fast read capacity (as mentioned elsewhere and in this article, its very fast for read heavy applications, which is most) you could take on an angle around that to get people "in…
> I think a strong - extremely strong - selling point is the point I made about "prebaked" data for your APIs. I think caches are an excellent use case for LiteFS early on. Sorry I didn't make that point in my previous reply. It's a good way to get benefits out of LiteFS without committing to it as your source of truth. Also related, Segment built a custom SQLite-based solution[1] for distributing out cached data tha…
With SQLite, we could just scale instances during peak demand (so you could read from different instances of the same data if there was a bottleneck) and scale back down again, without (!usually) losing the data.
It was a really complex - but fun - project all told, but its underpinnings were really simple.
How Segment approached the problem isn't dissimilar to how we did it, honestly.
The only thing that we (may) have done different is we had failover. If SQLite didn't respond our API layer could then talk directly to a database service to get the data. That was surprisingly complex to do.
Its entirely possible that even more robust setups than ours was when we did this would yield higher cost savings. We did this before we hit our next scale of customers, just to add a little more context