Message DB: Event Store and Message Store for PostgreSQL • Hacker News Board

Re: Message DB: Event Store and Message Store for PostgreSQL

#101

Post by AtlasBarfed » Thu, Dec 19, 2019, 5:47 PM UTC

simplecto wrote: post #25

sgk284 wrote: post #19

Earlier quoted context omitted.

At my previous startup we put everything we could into the database (work queues and pup/sub included). It was magical. It minimized complexity (moving parts, boundaries, etc...), everything was transactional, and a single dump of the database gave you a copy of the entire persistent state of your system at that moment (across all services). We had several services running, but any notion of persistent state was stor…

> Distributed systems are hard, avoid them until you can't. thank you. May I borrow this?

Yes, but why architect yourself into a more painful corner when you reach "can't"?

Re: Message DB: Event Store and Message Store for PostgreSQL

#102

Post by AtlasBarfed » Thu, Dec 19, 2019, 5:49 PM UTC

MaxBarraclough wrote: post #80

ianamartin wrote: post #55

Earlier quoted context omitted.

I can't watch that at work right now. But is it pretty much this? https://blog.cleancoder.com/uncle-bob/2012/05/15/NODB.html I've heard of him before but never really read anything much by him. Is this guy for real? He worked at one startup, got angry about a database, became a consultant, and screams about everyone being wrong about everything?

Yes he makes similar points there. A few scattered critical thoughts: * I don't see any problem with stored procedures. They can make good sense for, say, auditing, as well as for performance. * People describe their software systems as "Using Oracle" because it matters, not because their design is stupid. It tells me that Oracle skills are relevant, for instance. * NoSQL is generally, in my experience, awful and cha…

Yeah, I have changed my opinion of stored procs to :

If you have a proprietary database, stored procedures are awful. AHEM ORACLE.

Re: Message DB: Event Store and Message Store for PostgreSQL

#103

Post by AtlasBarfed » Thu, Dec 19, 2019, 5:56 PM UTC

kylecordes wrote: post #53

It is so interesting to see these problem solved again and again. Very important bit to understand, that many smart people (including perhaps some in this HN discussion) don't know yet: the semantics of an event store are meaningfully different from the semantics of a pub/sub system. If you pick up a pub/sub system and start trying to build an event store based system architecture with a set of CQRS-ES-style follower…

JGroups seems superficially related to lots of the cassandra functions, especially gossip.

Is it masterless, or does it handwave a lot of distribution problems like most java distributed projects by just using Zookeeper?

Re: Message DB: Event Store and Message Store for PostgreSQL

#104

Post by gunnarmorling » Sat, Dec 21, 2019, 4:25 PM UTC

kiwicopple wrote: post #30

hashhar wrote: post #26

Earlier quoted context omitted.

Look at embedded Debezium. https://debezium.io/docs/embedded/ Very similar to what you are doing but instead of Websockets you can decide where to send your data. The core-debezium sends it to Kafka.

I took a lot of inspiration from Debezium - at the time they required the wal2json plugin (not sure if that's still the case?). I didn't have an option to install the plugin and I mostly wanted a websocket-friendly implementation to replace Firebase. And if I'm 100% honest I just wanted to make something i thought was cool

By now the Debezium Postgres connector also works with pgoutput (coming with Postgres by default as of PG 10 and later).

Disclaimer: working on Debezium