SapphireDb – Open-Source Alternative to Firebase

Nice project! I had considered building something similar (actually just services discoverable via DHT, nothing concerning storage). Can you discuss here or in the README its resiliency toward bad actors? Specifically, how does it work in a p2p environment with untrusted clients? If it's still an unknown, just a note on whether it's even a threat model you're developing within would be helpful.

The hardest part was the real-time functionality. I ended up writing an elixir server that listens to Postgres’ built-in replication stream, converts it into JSON, then sends it over websockets.

For what it's worth, I have a silly side project goal of reimplementing Bitcoin on top of Redwood, just to prove it can provide the same (general) security guarantees. Not sure I'll get around to that any time soon, though :)

Anyway, here's what we do:

1. Users are identified by a public/private keypair (actually using Ethereum's implementation at the moment, just so I could focus on other stuff). All transactions must be signed by the sender, allowing recipients to verify the sender identities.

2. You can place "transaction validators" at any node in your state trees, and any transaction affecting the subtree under the validator will be checked by that validator. Currently, there's a "permissions" validator included that gives a simple way to control writes based on the Ethereum keypair I mentioned above. The current implementation of the permissions validator isn't very well fleshed out yet -- it just lets you specify a true/false write permission associated with a [user, keypath regex] pair -- but it's a decent placeholder for now, until I can get around to roles and policies.

3. You can also write custom transaction validators in Go/Lua/Javascript, which should make it trivial to implement just about any access control model you desire.

4. You can also create a "private" tree by explicitly specifying the set of users who are allowed to read from and write to that tree. The default Redwood node implementation does automatic peer discovery and keeps a list of peers whose identities/addresses have been verified. When it receives a transaction for a private tree, it only gossips that transaction to the tree's members (as opposed to its behavior with public trees, which is to gossip transactions to any peer who subscribes to that tree).

5. We also want secure persistent storage for transactions so that we can get high availability and redundancy without compromising the security model. To facilitate this, we allow you to configure the node to talk to what we're calling a "remote blind store" (essentially a key-value DB running on a separate piece of hardware, possibly off-site). The Redwood node encrypts transactions with a key the remote store doesn't possess and sends it over gRPC to the blind store.

6. Some applications will have a high volume of transaction data, and will want to be able to prune/merge/rebase that data to save space. To prevent bad actors from issuing malicious prune requests, the remote blind stores are going to implement their own p2p protocol involving threshold signatures. Prune requests will only be honored if a quorum of these blind stores sign off on them.

We've been thinking about this quite a bit, but not publicly. I'd love for someone to help us punch holes in it!