Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rxdb: A reactive database where you can subscribe to the result of a query

github.com

Post reply on HN
Print view View original

111–120 of 126 posts

Re: Rxdb: A reactive database where you can subscribe to the result of a query

#111

Post by ronmex »

pbowyer wrote: post #83
kiwicopple wrote: post #39

I have been developing something that provides this functionality for PostgreSQL: https://github.com/supabase/realtime It's still in very early stages (although I am using it in production for my company) It's very similar to Debezium (mentioned in another comment), but it's built with Phoenix (elixir), so great for listening via websockets. Basically the Phoenix server listens to PostgreSQL's replication functionali…

Interesting, thanks for sharing how you're doing it. Is there any mileage in doing this with triggers? I have a _very_ legacy system which needs caching adding. Rather than dig through the code to invalidate the cache every time a record is updated/deleted in 20+ tables, I am thinking that being able to listen to the SQL executed and invalidate the cache based on the tables involved would be a clean approach. But not…

You can use apache ignite as a caching layer over your db. (Or as a db itself). You can then register a ContinuousQuery to listen for changes. A grpc service can be stood up to stream these changes to other clients, even front end ones (via grpc-web).

Re: Rxdb: A reactive database where you can subscribe to the result of a query

#112

Post by cryptonector »

tjchear wrote: post #108
dsun180 wrote: post #41

Earlier quoted context omitted.

There is a big difference between the changestream of many SQL and noSQL databases, and what RxDB does. Having a stream of changes is useful but not the whole solution. RxDB is capable of using single document changes of a stream and recalculate the new results of an existing query. This saves you not only much IO performance but makes developing much easier. See https://rxdb.info/query-change-detection.html

I would love to use this just for the offline and query change detection capabilities alone - but the latter is currently beta and disabled by default. Is there a reason for that?

I do this sort of thing with triggers and table-based materializations of views.

Re: Rxdb: A reactive database where you can subscribe to the result of a query

#113

Post by markandrewj »

kiwicopple wrote: post #105
markandrewj wrote: post #103

Earlier quoted context omitted.

Just out of curiosity, are you moving off of Firebase because of technical limitations, or because of pricing?

Mainly because I want to keep my team small so it’s easier if we all commit to Postgres. But We also hard a hard time with Firebase’s filtering on sub collections (for their new Firestore)

Fair enough.

Re: Rxdb: A reactive database where you can subscribe to the result of a query

#114

Post by beaconstudios »

amelius wrote: post #94
beaconstudios wrote: post #80

Earlier quoted context omitted.

for my project it's very useful for collaborative features. Multiple people need to be able to edit the same graph structures at the same time, which would be difficult to scale out via a notify-and-poll architecture.

I've found that collaborative editing can be very confusing, and prefer a commit/merge step over simultaneous editing. I think all collaborative editing software should at least offer this kind of interface.

agreed - there's a branching model in the roadmap :)

Re: Rxdb: A reactive database where you can subscribe to the result of a query

#116

Post by ceejayoz »

discreteevent wrote: post #110
dsun180 wrote: post #55

Earlier quoted context omitted.

The definition of realtime is vague so saying that something is "not really realtime" just because it is not "realtime computing" is wrong.

In fairness it was a bit pretentious of whoever decided to call it real-time which was a word with a specific meaning and a very hard earned reputation. 'Live' might have been better.

"You see the updates in real time without having to refresh the page" seems like a perfectly reasonable statement.

I submit that being pedantic about such things is frequently its own form of pretension.

Re: Rxdb: A reactive database where you can subscribe to the result of a query

#117

Post by vlasky »

sergiotapia wrote: post #78
throwaway_bad wrote: post #3

Having client state just be a replica of server state solves so many problems I don't understand why the concept never caught on. Pouchdb/couchdb are still the only ones doing it afaik. Instead we have a bajillion layers of CRUD all in slightly different protocols just to do the same read or write to the database.

MeteorJS tried it and failed miserably. It doesn't scale.

Wrong. This is popular anti-Meteor FUD spread by people who don't know how to use its features properly or have the engineering/computer science background to design a system to be able to manage computational complexity or scalability.

In 2015, my business implemented a Meteor-based real-time vehicle tracking app utilising Blaze, Iron Router, DDP, Pub/Sub

Our Meteor app runs 24hrs/day and handles hundreds of drivers tracking in every few seconds whilst publishing real-time updates and reports to many connected clients. Yes, this means Pub/Sub and DDP.

This is easily being handled by a single Node.js process on a commodity Linux server consuming a fraction of a single core’s available CPU power during peak periods, using only several hundred megabytes of RAM.

How was this achieved?

We chose to use Meteor with MySQL instead of MongoDB. When using the Meteor MySQL package, reactivity is triggered by the MySQL binary log instead of the MongoDB oplog. The MySQL package provides finer-grained control over reactivity by allowing you to provide your own custom trigger functions.

Accordingly, we put a lot of thought into our MySQL schema design and coded our custom trigger functions to be selective as possible to prevent SQL queries from being needlessly executed and wasting CPU, IO and network bandwidth by publishing redundant updates to the client.

In terms of scalability in general, are we limited to a single Node.js process? Absolutely not - we use Nginx to terminate the connection from the client and spread the load across multiple Node.js processes. Similarly, MySQL master-slave replication allows us to spread the load across a cluster of servers.

For those using MongoDB, a Meteor package named RedisOplog provides improved scalability with the assistance of Redis's pub/sub functionality.

Re: Rxdb: A reactive database where you can subscribe to the result of a query

#118

Post by habitue »

dsun180 wrote: post #9
venantius wrote: post #7

I'm not sure why you would use this over something more battle-tested like RethinkDB which was also built with this use-case in mind.

RethinkDB is not offline first. It runs queries on the server-side, not on the client. There is horizonDB which can replicate with RethinkDB but the project is dead with no commits since the rethinkdb company gave up.

So I was one of the developers of RethinkDB's Horizon, and I'm really glad to see someone take the database change stream to RxJS concept to its logical conclusion.

This project looks really awesome, and very much has the shape we wished Horizon could have turned into

Re: Rxdb: A reactive database where you can subscribe to the result of a query

#119

Post by kiwicopple »

com2kid wrote: post #109
kiwicopple wrote: post #88

Earlier quoted context omitted.

I actually used Firebase at the start but this implementation is so that we can migrate 100% to Postgres. I agree though, Rethink is awesome

Make this good enough and you could very well cut into a huge chunk of Firebase's business model. I'd seriously love to have query subscriptions, especially if the library is robust enough for use on mobile. Ine large differentiator of Firebase's DB offerings is the subscription functionality, mobile and web, but using that means buying into NoSQL. A proper subscribable SQL database would be amazing. Of course the co…

Thanks for the comments. I agree, the conflict resolution of FB on mobile is pretty awesome.

FYI we are using this on our mobile (react native) in production, so I have no doubts it will be robust enough with a bit more polish

Post reply on HN