Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distributed SQLite: Paradigm shift or hype?

kerkour.com

Post reply on HN
Print view View original

21–30 of 165 posts

Re: Distributed SQLite: Paradigm shift or hype?

#21

Post by danappelxx »

tptacek wrote: post #19
danappelxx wrote: post #18

Earlier quoted context omitted.

Unless your database is in the browser, you are always going to be at mercy of network latencies talking to the backend.

You're just saying, even if all you were doing was fetching a static JSON blob from the memory of the frontend server, you'd still want load states, right? (That makes sense, I'm just checking my understanding.)

Yup, exactly. Phones change wifi networks, routers drop packets, load balancers get overloaded. Hard to fully eliminate tail latencies.

Re: Distributed SQLite: Paradigm shift or hype?

#23

Post by geenat »

WAL2 + BEGIN CONCURRENT

...to pretty much eliminate write contention. I know, it sucks these aren't built in yet.

Also if we got some sort of router + map reduce helper (Vitess, Citus -like) it'd make massively distributed SQLite a lot more viable. Setups that don't hammer a single master would make all the difference.

Postgres' main disadvantage at scale is all the additional machinery required (backups, failover, proxy, no DDL replication with built-in logical replication .... UGH!!), even when you're using Citus. Feels like it forces you into k8s with the amount of orchestration you need to run.

Re: Distributed SQLite: Paradigm shift or hype?

#26

Post by kragen »

evgpbfhnr wrote: post #22
kragen wrote: post #4

this page just says 'Forbidden'

Yeah, says Forbidden in firefox or curl but loads in chromium - welcome to the open web of browser interoperability etc etc where servers now just decide your user agent or tls handshake or whatever is fishy.

thank you!

Re: Distributed SQLite: Paradigm shift or hype?

#27

Post by simonw »

danappelxx wrote: post #21
tptacek wrote: post #19

Earlier quoted context omitted.

You're just saying, even if all you were doing was fetching a static JSON blob from the memory of the frontend server, you'd still want load states, right? (That makes sense, I'm just checking my understanding.)

Yup, exactly. Phones change wifi networks, routers drop packets, load balancers get overloaded. Hard to fully eliminate tail latencies.

The key here is to make a single API call to the backend which then runs 100+ SQL queries at once and combines the results into a single JSON response - that way you're only paying the network cost once.

See https://www.sqlite.org/np1queryprob.html

I've implemented GraphQL on top of SQLite and found it to be an amazingly good match, because the biggest weakness of GraphQL is that it makes it easy to accidentally trigger 100s of queries in one request and with SQLite that really doesn't matter.

Re: Distributed SQLite: Paradigm shift or hype?

#28

Post by hansvm »

kragen wrote: post #25
tptacek wrote: post #5

Earlier quoted context omitted.

I updated Chrome and reloaded.

[flagged]

Out of curiosity, did I stumble on an actual feud of sorts, or since they're a prolific commenter did they just not recognize you? Do you have links to the first four?

Re: Distributed SQLite: Paradigm shift or hype?

#29

Post by hifikuno »

hansvm wrote: post #28
kragen wrote: post #25

Earlier quoted context omitted.

[flagged]

Out of curiosity, did I stumble on an actual feud of sorts, or since they're a prolific commenter did they just not recognize you? Do you have links to the first four?

I, too, am curious. I have been searching comments for a bit but haven't found anything.

Re: Distributed SQLite: Paradigm shift or hype?

#30

Post by raggi »

geenat wrote: post #23

WAL2 + BEGIN CONCURRENT ...to pretty much eliminate write contention. I know, it sucks these aren't built in yet. Also if we got some sort of router + map reduce helper (Vitess, Citus -like) it'd make massively distributed SQLite a lot more viable. Setups that don't hammer a single master would make all the difference. Postgres' main disadvantage at scale is all the additional machinery required (backups, failover, p…

this does not eliminate write contention, it just moves a bunch of the problem into the application as now you have to resolve conflicts yourself
Post reply on HN