Ask HN: Best books on modern distributed systems

Post by skrtskrt » Thu, Sep 02, 2021, 7:35 PM UTC

playing_colours wrote: post #10

skrtskrt wrote: post #8

Designing Data Intensive Applications has treated me well, getting me through pretty much every system design interview I have had. I am going to move onto either * O'Rielly Designing Distributed Systems or * O'Rielly Architecture Patterns with Python Could anyone recommend either one? I am mainly a Python dev (with no particular love for Python), but that doesn't necessarily make me want to use the Python book more.…

O’Reilly Building Event-Driven Microservices is a solid book to continue. I would not recommend Architecture Patterns with Python. Some patterns there are questionable and introduce more complexity. Also, they talk about using Kafka and Service Bus in the book, but never mention an important issue you may face in real implementation: consistency in updating both a message bus and a database that usually is addressed…

Post by zeroc8 » Thu, Sep 02, 2021, 9:17 PM UTC

skrtskrt wrote: post #12

playing_colours wrote: post #10

Earlier quoted context omitted.

O’Reilly Building Event-Driven Microservices is a solid book to continue. I would not recommend Architecture Patterns with Python. Some patterns there are questionable and introduce more complexity. Also, they talk about using Kafka and Service Bus in the book, but never mention an important issue you may face in real implementation: consistency in updating both a message bus and a database that usually is addressed…

What do you think of avoiding implementing Outbox by just having things like Kafka Connect subscribe to the table changes in the database?

Post by playing_colours » Thu, Sep 02, 2021, 11:26 PM UTC

skrtskrt wrote: post #12

playing_colours wrote: post #10

Earlier quoted context omitted.

O’Reilly Building Event-Driven Microservices is a solid book to continue. I would not recommend Architecture Patterns with Python. Some patterns there are questionable and introduce more complexity. Also, they talk about using Kafka and Service Bus in the book, but never mention an important issue you may face in real implementation: consistency in updating both a message bus and a database that usually is addressed…

What do you think of avoiding implementing Outbox by just having things like Kafka Connect subscribe to the table changes in the database?

Post by lmarcos » Fri, Sep 03, 2021, 4:45 AM UTC

skrtskrt wrote: post #14

zeroc8 wrote: post #13

Earlier quoted context omitted.

One thing would be that you then are just syncing database as opposed to sending domain events, aren't you?

Good point, but you could have the sender/pusher process read from Kafka Connect and parse into a domain event before pushing. It sounds roundabout, but you're letting a battle-tested implementation handle the database-table-listening aspects with no special outbox implementation needed on the database write.

Post by playing_colours » Fri, Sep 03, 2021, 9:49 AM UTC

lmarcos wrote: post #17

skrtskrt wrote: post #14

Earlier quoted context omitted.

Good point, but you could have the sender/pusher process read from Kafka Connect and parse into a domain event before pushing. It sounds roundabout, but you're letting a battle-tested implementation handle the database-table-listening aspects with no special outbox implementation needed on the database write.

How do you handle domain events that are the composition of columns of different tables?