Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LiteFS

fly.io

Post reply on HN
Print view View original

41–50 of 158 posts

Re: LiteFS

#41

Post by tshaddox »

tptacek wrote: post #18
pphysch wrote: post #16

> Developing against a relational database requires devs to watch out for "N+1" query patterns, where a query leads to a loop that leads to more queries. N+1 queries against Postgres and MySQL can be lethal to performance. Not so much for SQLite. This is misleading AFAICT. The article(s) is actually comparing remote RDBMS to local RDBMS, not Postgres to SQLite. Postgres can also be served over a UNIX socket, removing…

Anyone want to chime in with the largest app they've deployed where prod Postgres was reachable over a Unix domain socket?

Are there significant limits to the size of an app that could be deployed alongside Postgres versus the size of an app that could be deployed alongside SQLite?

Re: LiteFS

#42

Post by woooooo »

pphysch wrote: post #29
woooooo wrote: post #23

Earlier quoted context omitted.

Isn't "n + 1" typically a degenerate case of ORM rather than a relational thing? The whole point of rdbms is you can do a join instead of iterated lookups.

Yeah most N+1 issues can be mitigated by optimizing the queries. You definitely don't need an ORM to get to N+1 hell, a for-loop in vanilla PHP will do.

Yeah, but you have to go out of your way to do that. If you're in touch with the SQL you're executing at all, the join is the obvious easy way to do it, it's not some arcane optimization.

Re: LiteFS

#43

Post by 0x457 »

pphysch wrote: post #36
mgkimsal wrote: post #34

Earlier quoted context omitted.

A for-loop in vanilla anything will do. Nothing special about PHP. Ruby, Python, Java, etc...

Of course. PHP is the one you will mainly encounter in the wilderness, though.

Most common place I saw N+1 is ActiveRecord (Ruby on Rails).

Because by default, no relationships are loaded and the way AR "loads" stuff makes is extremely easy to create N+M queries on 5 lines of ruby. Then, those people will tell you that you run out of database before you run out of ruby...

Re: LiteFS

#44

Post by clord »

pphysch wrote: post #35
benbjohnson wrote: post #31

Earlier quoted context omitted.

You could run Postgres over UNIX sockets although you will still get higher latency than SQLite's in-process model. Also, running a Postgres on every app instance on the edge probably isn't practical. Postgres has some great advanced features if you need them but it's also much more heavy weight. With LiteFS, we're aiming to easily run on low resource cloud hardware such as nodes with 256MB or less of RAM. I haven't…

SQLite almost certainly is the better edge RDBMS than Postgres, if only because it has less features taking up space. However, "local SQLite vs. remote Postgres/MySQL" remains a false dichotomy when talking about network latency.

It's an application design choice. It's perfectly reasonable to consider those two options when designing a system. The constraints of the system determine which is better for the application.

I'd pick a centralized network-reachable database with a strong relational schema for write-heavy applications, and a lighter in-process system for something that is mostly reads and where latency matters. It's not a false dichotomy — but more like a continuum that certainly includes both extremes on it.

Re: LiteFS

#45

Post by ignoramous »

asim wrote: post #7

10 years ago fly.io is the company I wanted to build. Something with massive technical depth that becomes a developer product. They're doing an incredible job and part of that comes down to how they evangelise the product outside of all the technical hackery. This requires so much continued effort. AND THEN to actually run a business on top of all that. Kudos to you guys. I struggled so much with this. Wish you nothi…

5 years ago, fly.io was basically poor man's deno.com / oven.sh [0]. In my (incorrect) opinion, tptacek changed fly.io's trajectory single-handedly.

[0] https://ghostarchive.org/varchive/r-1hXDvOoHA

Re: LiteFS

#46

Post by ignoramous »

toast0 wrote: post #27
theomega wrote: post #12

Does anyone else bump into the issue, that the fly.io website does not load if requested via IPv6 on Mac? I tried Safari, Chrome and curl and neither work: $ curl -v https://fly.io/blog/introducing-litefs/ * Trying 2a09:8280:1::a:791:443... * Connected to fly.io (2a09:8280:1::a:791) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem…

Could easily be a path MTU issue? I don't currently have working IPv6, but on TCP/IPv4, the server->client handshake packets I get back from fly.io are full length packets. If your tcpv6 MSS is set incorrectly, and some other not great things are happening in the path, then you might not be able to receive large packets.

> Could easily be a path MTU issue?

Perhaps: https://community.fly.io/t/ipv6-pmtud-issue/5081

Re: LiteFS

#47

Post by infogulch »

> To improve latency, we're aiming at a scale-out model that works similarly to Fly Postgres. That's to say: writes get forwarded to the primary and all read requests get served from their local copies.

How can you ensure that a client that just performed a forwarded write will be able to read that back on their local replica on subsequent reads?

Re: LiteFS

#48

Post by mwcampbell »

I wonder if using FUSE has had any appreciable impact on performance, particularly read performance. I ask because FUSE has historically had a reputation for being slow, e.g. with the old FUSE port of ZFS.

Re: LiteFS

#49

Post by clord »

I can imagine a database which tries to solve both of these domains.

A centralized database handles consistency, and vends data closures to distributed applications for in-process querying (and those closures reconcile via something like CRDT back to the core db).

Does this exist?

Re: LiteFS

#50

Post by shrubble »

tptacek wrote: post #18
pphysch wrote: post #16

> Developing against a relational database requires devs to watch out for "N+1" query patterns, where a query leads to a loop that leads to more queries. N+1 queries against Postgres and MySQL can be lethal to performance. Not so much for SQLite. This is misleading AFAICT. The article(s) is actually comparing remote RDBMS to local RDBMS, not Postgres to SQLite. Postgres can also be served over a UNIX socket, removing…

Anyone want to chime in with the largest app they've deployed where prod Postgres was reachable over a Unix domain socket?

It's a great way to deploy if you have the RAM and disk.

Bump up the shared buffers, have a multithreaded or pooled driver for your app server so connections are not being opened and closed all the time.

It really 'flies' under such a scenario.

Largest app was a big analytics and drug tracking application for a small pharmaceutical company. Not sure of the size but it was very reporting heavy.

Post reply on HN