Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LiteFS

fly.io

Post reply on HN
Print view View original

141–150 of 158 posts

Re: LiteFS

#141

Post by ignoramous »

benbjohnson wrote: post #14
no_wizard wrote: post #10

Earlier quoted context omitted.

Note: I apologize if this is overstepping, its hard to tell! I think a strong - extremely strong - selling point is the point I made about "prebaked" data for your APIs, since the entire strength of these SQLite based systems reside in their fast read capacity (as mentioned elsewhere and in this article, its very fast for read heavy applications, which is most) you could take on an angle around that to get people "in…

> I think a strong - extremely strong - selling point is the point I made about "prebaked" data for your APIs. I think caches are an excellent use case for LiteFS early on. Sorry I didn't make that point in my previous reply. It's a good way to get benefits out of LiteFS without committing to it as your source of truth. Also related, Segment built a custom SQLite-based solution[1] for distributing out cached data tha…

> I think caches are an excellent use case for LiteFS early on.

I mean, folks can do stuff like this on Fly with Redis backed by disk, too: https://fly.io/blog/last-mile-redis/

Re: LiteFS

#142

Post by benbjohnson »

mch82 wrote: post #131
benbjohnson wrote: post #5

Earlier quoted context omitted.

LiteFS/Litestream author here. You bring up a lot of great points that I'll try to address. > What I'd like to have seen is how this compares to things like rqlite or Cloudflare's D1 addressed directly in the article I think a post comparing the different options is a great idea. I'll try to summarize a bit here though. LiteFS aims to be an analogue to Postgres replication but with built-in failover. Postgres uses lo…

Are you adopting the TH3 approach for your codebase? One of the distinguishing things about SQLite is the DO-178B TH3. Not sure if any other open source database have that. http://www3.sqlite.org/th3.html

The TH3 test is proprietary and, IIRC, it might only be available to SQLite Consortium members and that's $125k/year. We do pay for SQLite support at Fly.io but not quite to that level.

We do have plans to run against the Tcl test suite[1] although most of that test suite is not applicable to LiteFS since it tests higher level constructs.

Since LiteFS acts on the raw pages, it really just functions similar to a VFS. Out of the 130K source lines of code in SQLite, only 4.8k are for the Unix VFS. As such, the testing coverage from the SQLite test suite mostly tests non-VFS code.

[1]: https://github.com/superfly/litefs/issues/17

Re: LiteFS

#143

Post by yencabulator »

pphysch wrote: post #35
benbjohnson wrote: post #31

Earlier quoted context omitted.

You could run Postgres over UNIX sockets although you will still get higher latency than SQLite's in-process model. Also, running a Postgres on every app instance on the edge probably isn't practical. Postgres has some great advanced features if you need them but it's also much more heavy weight. With LiteFS, we're aiming to easily run on low resource cloud hardware such as nodes with 256MB or less of RAM. I haven't…

SQLite almost certainly is the better edge RDBMS than Postgres, if only because it has less features taking up space. However, "local SQLite vs. remote Postgres/MySQL" remains a false dichotomy when talking about network latency.

There's still plenty of overhead in serializing data over a unix domain socket to a different process, waiting for that process to be scheduled, waiting for it to serialize a response, waiting for the client process to be scheduled. SQLite avoids all of that.

Re: LiteFS

#144

Post by yencabulator »

cbsmith wrote: post #105
simonw wrote: post #93

Earlier quoted context omitted.

The performance boost that matters most here is when your application reads from the database. Your application code is reading directly from disk there, through a very thin FUSE layer that does nothing at all with reads (it only monitors writes). So your read queries should mostly be measured in microseconds.

> So your read queries should mostly be measured in microseconds. You should check out the read latency for read-only requests over unix domain sockets with PostgreSQL. You tend to measure it in microseconds, and depending on circumstances it can be single-digit microseconds. Regardless of whether your FUSE logic does nothing at all, It sure seems like there's intrinsic overhead to the FUSE model that is very similar…

Yes, FUSE adds overhead. However,

1. kernel page cache fully removes the read overhead for in-memory pages

2. There's a FUSE_PASSTHROUGH mode that removes that overhead for all reads & writes. I haven't studied exactly what writes LiteFS needs to observe (just journal vs all data writes), but at least for reads it seems to pass them straight through. We could well submit a FUSE_PASSTHROUGH_READ patch to the kernel, and use that to remove all read overhead. The patch should be trivial, since the full FUSE_PASSTHROUGH mode is there already.

Disclaimer: I wrote the FUSE framework LiteFS uses, https://bazil.org/fuse

https://source.android.com/docs/core/storage/fuse-passthroug... https://lwn.net/Articles/674286/

Re: LiteFS

#145

Post by ignoramous »

stingraycharles wrote: post #68
ignoramous wrote: post #45

Earlier quoted context omitted.

5 years ago, fly.io was basically poor man's deno.com / oven.sh [0]. In my (incorrect) opinion, tptacek changed fly.io's trajectory single-handedly. [0] https://ghostarchive.org/varchive/r-1hXDvOoHA

What is @tptacek’s relation with fly.io, and how did he change it?

tptacek is a security researcher and principal at fly.io [0]. And as an outsider looking in, seems to have been the eng behind some of their defining features [1]:

- udp support

- container2vm overhaul

- private networks aka 6pn

- some key flyctl (cli) commands like flyctl ssh, flyctl proxy

- metrics

- litefs

- perhaps, the imminent overhaul of the orchestration layer (?)

- the upcoming authz layer

[0] https://archive.is/Zwzlh

[1] https://community.fly.io/u/thomas/activity/topics | https://fly.io/blog/author/thomas/

Re: LiteFS

#146

Post by benbjohnson »

ngrilly wrote: post #138
benbjohnson wrote: post #113

Earlier quoted context omitted.

It’s available as a file handle. If you have a database file named “db” then you can read “db-pos” to read the replication position. The position is a tuple of the TXID and the database checksum.

Can the app read it concurrently while it is updated by LiteFS?

Yes, the app can read it whenever. There's a brief lock obtained on the current database position internally to read it out. The position only gets updated at the end of the transaction.

Re: LiteFS

#147

Post by yencabulator »

ngrilly wrote: post #138
benbjohnson wrote: post #113

Earlier quoted context omitted.

It’s available as a file handle. If you have a database file named “db” then you can read “db-pos” to read the replication position. The position is a tuple of the TXID and the database checksum.

Can the app read it concurrently while it is updated by LiteFS?

Looks like LiteFS invalidates the cached file contents on every update of position. This means there is never stale data to be read.

(The client needs to make sure to consume that in a single pread/read syscall, or it could observe a sheared state.)

Disclaimer: I wrote the FUSE framework LiteFS uses, https://bazil.org/fuse

https://github.com/superfly/litefs/blob/52e269d4b04070690ce2... https://github.com/superfly/litefs/blob/52e269d4b04070690ce2... https://github.com/superfly/litefs/blob/a5cf33d1a3a91873d4ad...

Re: LiteFS

#148

Post by ignoramous »

tptacek wrote: post #123
ignoramous wrote: post #45

Earlier quoted context omitted.

5 years ago, fly.io was basically poor man's deno.com / oven.sh [0]. In my (incorrect) opinion, tptacek changed fly.io's trajectory single-handedly. [0] https://ghostarchive.org/varchive/r-1hXDvOoHA

No? No. No!

You won't change my mind with 3 No's. A fourth however...

Re: LiteFS

#149

Post by yencabulator »

ignoramous wrote: post #145
stingraycharles wrote: post #68

Earlier quoted context omitted.

What is @tptacek’s relation with fly.io, and how did he change it?

tptacek is a security researcher and principal at fly.io [0]. And as an outsider looking in, seems to have been the eng behind some of their defining features [1]: - udp support - container2vm overhaul - private networks aka 6pn - some key flyctl (cli) commands like flyctl ssh, flyctl proxy - metrics - litefs - perhaps, the imminent overhaul of the orchestration layer (?) - the upcoming authz layer [0] https://archiv…

> seems to have been the eng behind some of their defining features

> - litefs

That glory goes to https://news.ycombinator.com/user?id=benbjohnson

Re: LiteFS

#150

Post by yencabulator »

mwcampbell wrote: post #48

I wonder if using FUSE has had any appreciable impact on performance, particularly read performance. I ask because FUSE has historically had a reputation for being slow, e.g. with the old FUSE port of ZFS.

Often, the SQLite database would fit in RAM, and reads would be served straight from the page cache, with no overhead.

Disclaimer: I wrote the FUSE framework LiteFS uses, https://bazil.org/fuse -- and I also have some pending performance-related work to finish, there...

Post reply on HN