Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniCouchDB in Rust

garrensmith.com

Post reply on HN
Print view View original

31–40 of 47 posts

Re: MiniCouchDB in Rust

#31

Post by tehbeard »

xrd wrote: post #23
lukevp wrote: post #2

This is really cool, thanks for your work! I really love couchdb. I have seen tickets for document based ACLs on their github recently, I think this is the last feature they need to open up usage across many more domains. The core app works great for replication between dbs and to the edge using pouchdb, but almost everyone is turned off by the database-per-user that is required to use it. If they solve that I think…

This is interesting to me. One of the things that really turned me off from firebase is the rules system where you have to build an ACL there. It's been a few years since I've played heavily with it, but I really disliked trying to essentially build my authorization system into a backend which I couldn't run locally and was primarily edited in their web console. For me, writing a proxy that sits in front of couchdb i…

_changes should be available in an event stream format for continuous consumption, and you can apply filter functions to the feed?

Re: MiniCouchDB in Rust

#32

Post by xrd »

tehbeard wrote: post #31
xrd wrote: post #23

Earlier quoted context omitted.

This is interesting to me. One of the things that really turned me off from firebase is the rules system where you have to build an ACL there. It's been a few years since I've played heavily with it, but I really disliked trying to essentially build my authorization system into a backend which I couldn't run locally and was primarily edited in their web console. For me, writing a proxy that sits in front of couchdb i…

_changes should be available in an event stream format for continuous consumption, and you can apply filter functions to the feed?

It is, but I am using the global changes feed (so I don't need to create a process for each database which doesn't scale for me). And you have to manually create that database to have it work. And I have to filter based on my own code, I just wish there was a way to subscribe to high level events. In general it works fine but I keep thinking there could be a better way.

Re: MiniCouchDB in Rust

#33

Post by dmix »

Both helpful git repos for learning how to architect async Rust apps using Tokio. `mini-redis` is especially well documented (I understand the current MiniCouchDB is still early!). Thanks for sharing.

Re: MiniCouchDB in Rust

#34

Post by matlin »

lukevp wrote: post #28
daleharvey wrote: post #20

Earlier quoted context omitted.

This is an architecture I use a lot. The reverse proxy is there to implement the replication, having your application understand the replication protocol is a big ask. Instead, your app takes http requests, handles the ones it is supposed to, and forwards database requests directly to (c|p)ouchdb (after checking auth) https://github.com/daleharvey/noted/blob/master/index.js is a very simple example of how it can work

We really shouldn't be writing our own authentication layers (or anything security-related for that matter) unless necessary. Case in point, this example code has a massive security issue that allows anyone to impersonate without tokens if there is an active authentication request open. Hopefully no one has used this example code to build a production system that has real user data. This is a perfect example of why t…

You're probably write about writing your own authentication layer but it still shouldn't be part of CouchDB or PouchDB. A better solution is for some OSS project to build a standard proxy that applies the document level authorization that everyone is asking for. No reason for it to be built-in.

Re: MiniCouchDB in Rust

#35

Post by xrd »

lukevp wrote: post #12
foolmeonce wrote: post #6

Earlier quoted context omitted.

It seems to me like the pieces never really came together at the right time to make couchapps and I find that a bit sad.. I don't really like having apache in the middle, but ssl wasn't ready, then references to external services went away, now html view designs drop out just as the JS support is modernized.. I think it's generally a problem that taken alone these features aren't useful until they are all together go…

Great perspective. Totally agree. We are using nginx to proxy to couchdb and do the ssl termination, and it’s great that we can proxy the DB traffic like this since it’s all just http, but couch has always felt one step away from being an absolutely killer dominator of the MBaaS space to me. It’s like a self hosted firestore, except it has no story around auth. If it also had a way to trigger user code on the server…

This is a great comment.

In all fairness, they did have an auth story, right, and recent documentation suggests they reconsidered that path and now suggest keeping it out of couchdb. So, to me, this says auth was something they never could get right because it is complicated. And I took that a step further to think that it's better to have it outside because you can use any auth solution you want, instead of what the couchdb people felt were the best way to do it (it's smart that they realized they got it wrong and changed course).

I'm confused why everyone here seems to think reverse proxies and auth proxies are complicated. Isn't it the case that all apps of any complexity are a bunch of small services wired together behind a proxy? My auth proxy is all of 50 lines of code, my reverse proxy is 8 lines in nginx conf and it's all held together with a docker compose file that is declarative and works locally as well as on my production server.

Re: MiniCouchDB in Rust

#36

Post by lukevp »

matlin wrote: post #34
lukevp wrote: post #28

Earlier quoted context omitted.

We really shouldn't be writing our own authentication layers (or anything security-related for that matter) unless necessary. Case in point, this example code has a massive security issue that allows anyone to impersonate without tokens if there is an active authentication request open. Hopefully no one has used this example code to build a production system that has real user data. This is a perfect example of why t…

You're probably write about writing your own authentication layer but it still shouldn't be part of CouchDB or PouchDB. A better solution is for some OSS project to build a standard proxy that applies the document level authorization that everyone is asking for. No reason for it to be built-in.

I would agree if couchdb wasn't positioning itself as being exposed directly on the internet. If CouchDB is meant to be proxied behind another system / auth stack, then why does it have CORS support and cookie auth built in?

Both of those features are purely a client-side concern and exist because the original intent of CouchDB was real-time replication to browsers. Otherwise the proxy could do the CORS as well as the authentication and CouchDB would only require an http-level authentication pattern (like basic auth).

Having said that, I do agree that this should be compartmentalized and the end user should be able to pick and choose what features they want to allow, but I don't think that this should continue to be a separate concern that everyone is building themselves, it should be a first-party solution.

Re: MiniCouchDB in Rust

#37

Post by lukevp »

xrd wrote: post #35
lukevp wrote: post #12

Earlier quoted context omitted.

Great perspective. Totally agree. We are using nginx to proxy to couchdb and do the ssl termination, and it’s great that we can proxy the DB traffic like this since it’s all just http, but couch has always felt one step away from being an absolutely killer dominator of the MBaaS space to me. It’s like a self hosted firestore, except it has no story around auth. If it also had a way to trigger user code on the server…

This is a great comment. In all fairness, they did have an auth story, right, and recent documentation suggests they reconsidered that path and now suggest keeping it out of couchdb. So, to me, this says auth was something they never could get right because it is complicated. And I took that a step further to think that it's better to have it outside because you can use any auth solution you want, instead of what the…

Thanks for the shout out.

The problem with auth is in the authorization realm, not the authentication realm. It's super easy to do authentication in nginx for example using client certificates, basic auth, api keys, or in your app layer via checking a separate DB or cache for tokens, say via a JWT.

However, it's not trivial to say user "a" owns document "b" but not document "c", and that user "a", in fact, shouldn't even know that document "c" exists at all, while also maintaining the replication that CouchDB has built in.

What we want is one single DB that has all documents, and can replicate all documents on the server level with another DB, but can expose a user-specific changes feed that replicates only data that user has access to. I should be able to grant / revoke access to a document at any time and have it propagate, and all documents should be owned only by the creating user by default.

The choice they are requiring us to make is to shard data ourselves by user (which is unusable in the context of 2 users sharing data), or implement a separate layer that can understand CouchDB replication and can do the filtering of the changes feed as well as the write access to ensure that the documents are restricted correctly.

Take this a step further and now that I have to implement my own authentication AND authorization outside of CouchDB and protect it from the end user directly accessing it, and you could ask the question of why do I even need CouchDB then? Why not just speak CouchDB replication protocol on top of my own auth database and store documents there too?

Re: MiniCouchDB in Rust

#38

Post by xrd »

lukevp wrote: post #37
xrd wrote: post #35

Earlier quoted context omitted.

This is a great comment. In all fairness, they did have an auth story, right, and recent documentation suggests they reconsidered that path and now suggest keeping it out of couchdb. So, to me, this says auth was something they never could get right because it is complicated. And I took that a step further to think that it's better to have it outside because you can use any auth solution you want, instead of what the…

Thanks for the shout out. The problem with auth is in the authorization realm, not the authentication realm. It's super easy to do authentication in nginx for example using client certificates, basic auth, api keys, or in your app layer via checking a separate DB or cache for tokens, say via a JWT. However, it's not trivial to say user "a" owns document "b" but not document "c", and that user "a", in fact, shouldn't…

I was thinking about my comment, and thought: "I need to ask if you mean authz or authn?" That's funny you said it first.

All those things you note are important, and I just can't see how CouchDB would get those things right inside CouchDB.

My impressions with Firebase/Firestore were that they tried to do that with their "rules" system, and it was a not-quite-JS declarative system that relied on you understanding the non-standard parts of their auth system and the things they exposed. I always felt like this was going to be a huge hole in my app and I would have no way to validate all the edge cases. I feel like this is a complicated beast to do in a generic way and CouchDB was smart to leave it to the app developer, rather than the devops/sysadmin role.

Aren't you being a little stingy with your appreciation for the sync part of CouchDB/PouchDB and a little bit overblown with your worries about understanding the CouchDB "protocol?"

Replication and sync are really challenging problems even if you just think about sharing data in two places, and when you start having your JS code deal with revisions, it gets messy really quickly. That's what appealed to me about PouchDB was never having to really think about sync, other than how to handle conflicts.

But, the CouchDB protocol is just HTTP. And, making a proxy to talk to that is as simple as importing a http proxy module. You are just responsible for your authz logic, which is hard, but at least you can make it exactly the way you want it. I don't really see how mapping your authn logic onto standard HTTP verbs like POST, GET, PUT is that complicated.

Having said all this, you clearly have thought through this stuff deeply, so I'm very interested in hearing your thoughts here because I'm sure my comments are wrong past the surface.

Re: MiniCouchDB in Rust

#39

Post by xrd »

xrd wrote: post #38
lukevp wrote: post #37

Earlier quoted context omitted.

Thanks for the shout out. The problem with auth is in the authorization realm, not the authentication realm. It's super easy to do authentication in nginx for example using client certificates, basic auth, api keys, or in your app layer via checking a separate DB or cache for tokens, say via a JWT. However, it's not trivial to say user "a" owns document "b" but not document "c", and that user "a", in fact, shouldn't…

I was thinking about my comment, and thought: "I need to ask if you mean authz or authn?" That's funny you said it first. All those things you note are important, and I just can't see how CouchDB would get those things right inside CouchDB. My impressions with Firebase/Firestore were that they tried to do that with their "rules" system, and it was a not-quite-JS declarative system that relied on you understanding the…

And, this might be an interesting open source project to collaborate on. I'd be very game for that. xrdawson at the google owned mail system. :)

Re: MiniCouchDB in Rust

#40

Post by tgtweak »

lukevp wrote: post #30
karmelapple wrote: post #21

Earlier quoted context omitted.

We moved to CouchDB because we didn’t want to reinvent syncing. We wanted a tech that could handle the online and offline case well, and CouchDB and some supporting libraries (PouchDB, some native mobile OS libraries) helped with this. Syncing was a solved problem in the CouchDB world, so we moved to CouchDB. Access controls also seem like they should be a “solved problem,” something we can adopt that works well with…

That RFC/PR adds ACLs at the document level to CouchDB, but it has been open for a year now. I hope they merge it as well. Until that PR the party line has always been to do ACL outside the DB like you would with any other DB, which really doesn’t make much sense to me.

Have a look at the ($3000/core) enterprise version of Microsoft SQL server and check the field and row level permissions with full inherited RBAC... It's a feature that has some need in the entreprise world. Ignoring these features because you can do it in the application is a bit of a sabotage... take the logic far enough and you'll eventually get to the conclusion that maybe you should build your own DB too.
Post reply on HN