Marmot: Multi-writer distributed SQLite based on NATS
1–10 of 50 posts
Re: Marmot: Multi-writer distributed SQLite based on NATS
#2Something that wasn't clear to me from the README: how does this handle duplicate IDs?
If I have a table with a string primary key and I insert a row in one node with ID "hello" and do the same thing (but with different column data) on another node, what happens?
Re: Marmot: Multi-writer distributed SQLite based on NATS
#3Something that wasn't clear to me from the README: how does this handle duplicate IDs? If I have a table with a string primary key and I insert a row in one node with ID "hello" and do the same thing (but with different column data) on another node, what happens?
I'm not 100% sure of this but from the README section on race conditions:
"the last writer will always win. This means there is NO serializability guarantee of a transaction spanning multiple tables. This is a design choice, in order to avoid any sort of global locking, and performance."
So it sounds like in your example, whichever node writes last with a given primary key will be the data you'll end up with.
Re: Marmot: Multi-writer distributed SQLite based on NATS
#4I wonder if this kind of a setup would be useful for federated protocols. Mastodon and Farcaster both maintain websocket connections open from all servers to all servers.
Re: Marmot: Multi-writer distributed SQLite based on NATS
#5Something that wasn't clear to me from the README: how does this handle duplicate IDs? If I have a table with a string primary key and I insert a row in one node with ID "hello" and do the same thing (but with different column data) on another node, what happens?
I'm not 100% sure of this but from the README section on race conditions: "the last writer will always win. This means there is NO serializability guarantee of a transaction spanning multiple tables. This is a design choice, in order to avoid any sort of global locking, and performance." So it sounds like in your example, whichever node writes last with a given primary key will be the data you'll end up with.
Based on their description of how it works with jetstream i think last insert will fail because it will see the row already exists in jetstream
Within a single table they have tx serializability, just not with multiple tables
Re: Marmot: Multi-writer distributed SQLite based on NATS
#6Something that wasn't clear to me from the README: how does this handle duplicate IDs? If I have a table with a string primary key and I insert a row in one node with ID "hello" and do the same thing (but with different column data) on another node, what happens?
In what context would you use a single string as the ID that would be subject to collisions (instead of including a user id as part of the ID)?
Re: Marmot: Multi-writer distributed SQLite based on NATS
#7I can totally see some junior dev who doesn't fully grok transactions (and not reading/understanding the "What happens when there is a race condition?" section), trying to deploy something like this and scratching their head why it falls over completely in production. The site worked fine when it was just me testing it!
This is a neat proof of concept and I encourage experimentation. But, if you're developing something, please just use postgres, and don't try to cobble things together with something like this.
Edit: already seeing the downvotes. Yes... classic HN... anything that goes against plain old sanity is punished.
Re: Marmot: Multi-writer distributed SQLite based on NATS
#8I can totally see some junior dev who doesn't fully grok transactions (and not reading/understanding the "What happens when there is a race condition?" section), trying to deploy something like this and scratching their head why it falls over completely in production. The site worked fine when it was just me testing it! This is a neat proof of concept and I encourage experimentation. But, if you're developing somethi…
Postgres transactions are not SERIALIZABLE by default either. Though, yes, Postgres defaults will catch some more conditions than this project will.
I think the important thing is to encourage devs to understand transactions in the first place.
Re: Marmot: Multi-writer distributed SQLite based on NATS
#9I can totally see some junior dev who doesn't fully grok transactions (and not reading/understanding the "What happens when there is a race condition?" section), trying to deploy something like this and scratching their head why it falls over completely in production. The site worked fine when it was just me testing it! This is a neat proof of concept and I encourage experimentation. But, if you're developing somethi…
Postgres transactions are not SERIALIZABLE by default either. Though, yes, Postgres defaults will catch some more conditions than this project will. I think the important thing is to encourage devs to understand transactions in the first place.
[deleted]
Re: Marmot: Multi-writer distributed SQLite based on NATS
#10Something that wasn't clear to me from the README: how does this handle duplicate IDs? If I have a table with a string primary key and I insert a row in one node with ID "hello" and do the same thing (but with different column data) on another node, what happens?
In what context would you use a single string as the ID that would be subject to collisions (instead of including a user id as part of the ID)?
People do all sorts of weird things with databases, and if you want to run existing software against a database (as opposed to greenfield development deliberately targeting Marmot) you need to understand what will happen in different cases.