Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anti-patterns in event-driven architecture

codeopinion.com

Post reply on HN
Print view View original

111–120 of 171 posts

Re: Anti-patterns in event-driven architecture

#111

Post by rswail »

chenster wrote: post #107
rswail wrote: post #63

Earlier quoted context omitted.

Wrote a public transport ticketing system that processes 100-200K+ trips/day with sub-second push of notification to mobiles of trip/payments. Event driven and CQRS "entities" made logic and processing much easier to create/test/debug. Primary issues: 1. Making sure you focus on the "Nouns" (entities) not the "Verbs". 2. Kafka requiring polling for consumers sucks if you want to "scale to zero". 3. Sharding of event…

Can you elaborate #1 Nouns over Vers?

A lot of people focus on the process instead of the participating entities.

The focus when designing the system should be on the entities (Customer, Payment, Bill, Order, Inventory) instead of the processes (ordering, billing, fulfillment). I summarize that by saying "Nouns over Verbs".

The state of each of the entities is affected by the processes, but the effect happens from changes in other entities, Customers place an Order. Customers get a Bill for the Order, Customers make a Payment, etc.

The states of each of these entities is independent of the others and reacts/changes only as a result of two things, either an external "Command", or an "Event".

Commands are events that occur outside of the system boundary, usually visible as part of an API (if RESTful) that uses POST/PUT/DELETE or they are imperatives from one entity to another.

Commands are imperatives, Place Order, Pay Bill, Fulfill Order, etc.

Events are records of occurrences in the system, expressed in the past tense and are immutable. Order Placed, Bill Paid, Order Fulfilled.

Customers place an Order by POSTing to /orders (or potentially /customers/uuid/orders).

Events are generated from entities inside the system. (Order being placed generates an order_placed event).

The difference is that by focussing on the entities, and their state, independent of other entities, the entities can be created, tested, installed, evolved independently of other entities in the system.

The thinking about them is simplified and focussed, they are naturally decoupled because they can only find out about other entities by inquiring or affect other entities by generating a Command or an Event.

Any events they generate are processed asynchronously and can have multiple consumers.

Re: Anti-patterns in event-driven architecture

#112

Post by davedx »

YZF wrote: post #86
junto wrote: post #66

Earlier quoted context omitted.

You need to improve your telemetry to feel the benefits. I can trace all the way through multiple services easily on a simple detailed flame graph in our systems. https://www.datadoghq.com/knowledge-center/distributed-traci... Unless you have a single monolith, you’re going to face issues with versioning whether it’s event based or API based. In each case you can usually add new properties to a message, but you can’t…

There are many examples of event sourcing but perhaps the most classical one is the bank account. It's not just about auditing, it's also about transactionality and atomicity. If you want to withdraw $5 from your account, the traditional approach of locking, updating everything, unlocking (or in other words wrapping everything in a transaction) doesn't scale as well as the notion that you just record the transaction…

Do you know for a fact that banks use event sourcing for transactions? I thought they were extremely eventually consistent

Re: Anti-patterns in event-driven architecture

#113

Post by YZF »

davedx wrote: post #112
YZF wrote: post #86

Earlier quoted context omitted.

There are many examples of event sourcing but perhaps the most classical one is the bank account. It's not just about auditing, it's also about transactionality and atomicity. If you want to withdraw $5 from your account, the traditional approach of locking, updating everything, unlocking (or in other words wrapping everything in a transaction) doesn't scale as well as the notion that you just record the transaction…

Do you know for a fact that banks use event sourcing for transactions? I thought they were extremely eventually consistent

I'm pretty sure different banks use vastly different architectures. Some run nightly batch jobs on mainframes that are written in COBOL. But alas, I've not worked for any bank, this is just a commonly used example. I am willing to bet the transaction log, or ledger, is indeed a very common approach since that's also common in accounting.

EDIT: Also event sourcing would typically be eventually consistent. I imagine for some banking applications a stronger consistency guarantee might be required, e.g. to prevent you from withdrawing the $100 in your account multiple times.

Re: Anti-patterns in event-driven architecture

#114

Post by chipdart »

junto wrote: post #66
arwhatever wrote: post #13

I often hear the argument in favor of event-driven architecture that you can work on one part of a system in isolation without having to consider the other parts, and then I get assigned some task which requires me to consider the entire system operation, now with events that are harder to trace than function calls would have been. Now when people argue “because decoupling,” I hear, “You don’t get as much notificatio…

You need to improve your telemetry to feel the benefits. I can trace all the way through multiple services easily on a simple detailed flame graph in our systems. https://www.datadoghq.com/knowledge-center/distributed-traci... Unless you have a single monolith, you’re going to face issues with versioning whether it’s event based or API based. In each case you can usually add new properties to a message, but you can’t…

> I can trace all the way through multiple services easily on a simple detailed flame graph in our systems.

That's not exactly an obscure feature exclusive to datadog. From the top of my head, both AWS and Azure support distributed tracing with dedicated support for visualization in their x-ray and application insights services.

Re: Anti-patterns in event-driven architecture

#115

Post by pyrale »

arwhatever wrote: post #13

I often hear the argument in favor of event-driven architecture that you can work on one part of a system in isolation without having to consider the other parts, and then I get assigned some task which requires me to consider the entire system operation, now with events that are harder to trace than function calls would have been. Now when people argue “because decoupling,” I hear, “You don’t get as much notificatio…

> now with events that are harder to trace than function calls

Same issue as microservices: there are people who want to use the paradigm but not do the investment in monitoring/tooling.

Re: Anti-patterns in event-driven architecture

#116

Post by Jenk »

chipdart wrote: post #114
junto wrote: post #66

Earlier quoted context omitted.

You need to improve your telemetry to feel the benefits. I can trace all the way through multiple services easily on a simple detailed flame graph in our systems. https://www.datadoghq.com/knowledge-center/distributed-traci... Unless you have a single monolith, you’re going to face issues with versioning whether it’s event based or API based. In each case you can usually add new properties to a message, but you can’t…

> I can trace all the way through multiple services easily on a simple detailed flame graph in our systems. That's not exactly an obscure feature exclusive to datadog. From the top of my head, both AWS and Azure support distributed tracing with dedicated support for visualization in their x-ray and application insights services.

I doubt GP was suggesting it is unique to DD.

Re: Anti-patterns in event-driven architecture

#118

Post by lz400 »

candiddevmike wrote: post #3

Can someone share some long term event driven success stories? Almost everything you see online is written by consultants or brand new, greenfield implementations, curious how long these systems last.

AFAIK almost every stock market order processing system is event driven, and they are all usually very old systems that have been successfully running for years. I've seen some implementation in investment banks, what you're usually told is that most exchanges and banks run similar architectures. The reason for this is partially that FIX, the protocol for electronic orders in markets is event based.

Re: Anti-patterns in event-driven architecture

#119

Post by pyrale »

majormajor wrote: post #105
trevor-e wrote: post #47

Earlier quoted context omitted.

Like anything it can be abused and sometimes folks go overboard with turning everything into an event. However, when done right, it is really amazing to work with. As an event producer as long as you follow reasonable backwards-compatibility best practices then you should be pretty safe from breaking things downstream. As a consumer, follow defensive programming and allow for idempotency in case you need to reprocess…

> As an event producer as long as you follow reasonable backwards-compatibility best practices then you should be pretty safe from breaking things downstream. That can protect you from "downstream can't even read the message anymore" but it doesn't help you with the much more common "downstream isn't doing the right thing with the message anymore" problem. Schema evolution is kinda like schema'd RPC calls vs plain JS…

Data consumers in unknown teams is a nightmare regardless of the architecture.

Events sent for readership you can’t control are ideally of the type «x changed», and the consumer must then fetch data on the relevant endpoint.

That or the company must have serious versioning policies.

Re: Anti-patterns in event-driven architecture

#120

Post by setr »

ninkendo wrote: post #28
candiddevmike wrote: post #3

Can someone share some long term event driven success stories? Almost everything you see online is written by consultants or brand new, greenfield implementations, curious how long these systems last.

Chiming in with another “no” here. We adopted a message bus/event-driven architecture when moving a very popular piece of software from the cloud, to directly on the user’s device… it was a disaster IMO. The core orchestration of the system was done via events on the bus, and nobody had any idea what was happening when a bug occurred. People would pass bugs around, “my code did the right thing given the event it got”…

> (Wild tangent: holy shit is hard to get iOS to accept “do process” in a sentence. I edited that paragraph at least 30 times, no joke, trying every trick I could to stop it correcting it to “due process”. I almost gave up. I used to defend autocorrect but holy shit that was a nightmare.)

can you not just pick the original spelling in the autocomplete menu above the keyboard?

Post reply on HN