Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LiteFS

fly.io

Post reply on HN
Print view View original

131–140 of 158 posts

Re: LiteFS

#131

Post by mch82 »

benbjohnson wrote: post #5
no_wizard wrote: post #2

This is distributed SQLite 3, running (I assume at least partially managed?) LiteFS[5] for you. Which is pretty cool! What I'd like to have seen is how this compares to things like rqlite[1] or Cloudflare's D1[2] addressed directly in the article That said, I think this is pretty good for things like read replica's. I know the sales pitch here is as a full database, and I don't disagree with it, and if I was starting…

LiteFS/Litestream author here. You bring up a lot of great points that I'll try to address. > What I'd like to have seen is how this compares to things like rqlite or Cloudflare's D1 addressed directly in the article I think a post comparing the different options is a great idea. I'll try to summarize a bit here though. LiteFS aims to be an analogue to Postgres replication but with built-in failover. Postgres uses lo…

Are you adopting the TH3 approach for your codebase? One of the distinguishing things about SQLite is the DO-178B TH3. Not sure if any other open source database have that. http://www3.sqlite.org/th3.html

Re: LiteFS

#133

Post by [deleted] »

cbsmith wrote: post #108
WorldMaker wrote: post #97

Earlier quoted context omitted.

It's an "eventual consistency" transaction scheme for SQLite: reads happen locally and immediately with the nearest edge copy available, it's only writes that need to be forwarded to a (potentially far away) primary DB over the network somewhere, and transactions will then take some amount of network time to fan back out to edges, so eventually the edges will read the latest transactions, but during that fan out will…

Local for the application code on the node you're talking to (just like the read would be local for a Postgres process running on the node), but there's still a network trip to get to the node, no? Even if the reads are happening locally, if they're going through FUSE (even a "thin" pass-through that does nothing), that means they're getting routed from kernel space to a FUSE daemon, which means you're still doing IP…

[deleted]

Re: LiteFS

#134

Post by tptacek »

kjeetgill wrote: post #132
tptacek wrote: post #33

Earlier quoted context omitted.

Yes, that is cheating.

I don't have any context around this: why is that cheating?

Because the point of the Unix domain socket (in this argument) is eliminating network round trips.

Re: LiteFS

#136

Post by randomdata »

woooooo wrote: post #23
pphysch wrote: post #16

> Developing against a relational database requires devs to watch out for "N+1" query patterns, where a query leads to a loop that leads to more queries. N+1 queries against Postgres and MySQL can be lethal to performance. Not so much for SQLite. This is misleading AFAICT. The article(s) is actually comparing remote RDBMS to local RDBMS, not Postgres to SQLite. Postgres can also be served over a UNIX socket, removing…

Isn't "n + 1" typically a degenerate case of ORM rather than a relational thing? The whole point of rdbms is you can do a join instead of iterated lookups.

It is a result of representing application state as complex entity structures. I am not sure I would call it degenerate. n+1 would be quite natural if relational databases were theoretically pure. It only falls down in the real world because we accept deviation from the theory to allow implementation.

Joins are one possible workaround to the n+1 problem. However, they push the problem into the application to unpack, which detracts from having a nice high level abstraction in the database for fetching data. The many queries of n+1 would be ideal if you had an ideal database. You are querying on behalf of many different entities so many different queries is conceptually sound. But since we don't have ideal databases...

That's not to say that joins don't have their place. In the relational model, joins are essential. But the n+1 problem happens when you are not working in the relational model.

Re: LiteFS

#137

Post by Existenceblinks »

The pain is that this approach is suitable on VPS/IaaS where disk volume is supported. As a solo dev, I only use PaaS kind of infra, there are just a few PaaS i'm aware of that support attachable disk. Fly, Render, .. nothing else?

Re: LiteFS

#138

Post by ngrilly »

benbjohnson wrote: post #113
simonw wrote: post #111

Earlier quoted context omitted.

Is that transaction ID made available to SQLite clients? As a custom SQL function or a value in a readable table somewhere?

It’s available as a file handle. If you have a database file named “db” then you can read “db-pos” to read the replication position. The position is a tuple of the TXID and the database checksum.

Can the app read it concurrently while it is updated by LiteFS?

Re: LiteFS

#139

Post by kjeetgill »

tptacek wrote: post #134
kjeetgill wrote: post #132

Earlier quoted context omitted.

I don't have any context around this: why is that cheating?

Because the point of the Unix domain socket (in this argument) is eliminating network round trips.

Oh gotcha, I assumed they meant all 3 we're on one host. I'm not too familiar with the pg ecosystem.

Re: LiteFS

#140

Post by presentation »

My dream would be if this supported geo-partitioning. In my field people are pretty sensitive about GDPR so would love to box in EU PII in EU servers.
Post reply on HN