Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anti-patterns in event-driven architecture

codeopinion.com

Post reply on HN
Print view View original

81–90 of 171 posts

Re: Anti-patterns in event-driven architecture

#81

Post by zdragnar »

Osiris wrote: post #58
arwhatever wrote: post #13

I often hear the argument in favor of event-driven architecture that you can work on one part of a system in isolation without having to consider the other parts, and then I get assigned some task which requires me to consider the entire system operation, now with events that are harder to trace than function calls would have been. Now when people argue “because decoupling,” I hear, “You don’t get as much notificatio…

What I like about event driven is that you don't even need to know if anyone is listening to or cares about your event. And as a consumer, many independent tasks can be triggered by the same event. I'm working on a system right now and because of events, it's very easy for me to write a handler for when a certain type of record is created in the database. My feature depends on knowing that new record was made so we c…

> you don't even need to know if anyone is listening to or cares about your event.

Right up until you need to change something about the event because the business logic it represents has changed. Then you suddenly need to track down all the systems that have been relying on it, including that one that nobody knows anything about and always forgets exists because some guy decided to implement the service in erlang and nobody who ever touched it even works at the company anymore.

Re: Anti-patterns in event-driven architecture

#82

Post by sattoshi »

mrkeen wrote: post #71
arwhatever wrote: post #13

I often hear the argument in favor of event-driven architecture that you can work on one part of a system in isolation without having to consider the other parts, and then I get assigned some task which requires me to consider the entire system operation, now with events that are harder to trace than function calls would have been. Now when people argue “because decoupling,” I hear, “You don’t get as much notificatio…

> now with events that are harder to trace than function calls would have been I don't know how this could be true. Events are things - nouns which can be backed-up, replicated, stored, queried, rendered, indexed and searched over.

How is it not true? Instead of tracking data and function calls over a unified stacktrace, you track Things and Messages over databases, queues, and logs —- none of which you can trivially attach a debugger to.

I generally like event-driven architecture, but I need to admit that debuggability is sacrificed where it matters most.

Re: Anti-patterns in event-driven architecture

#83

Post by ramchip »

mrkeen wrote: post #72
nitwit005 wrote: post #54

Earlier quoted context omitted.

I've seen successful, but flawed usage. Every use I've seen sent events after database transactions, with the event not part of the transaction. This means you can get both dropped events, and out of order events. My current company has analytics driven by a system like that. I'm sure there's some corrupted data as a result. The main issue being people just don't know how to build and test distributed systems.

I had an interview where I was asked how I would guarantee that an event happened in addition to a database update (transactionally). It sounded kind of impossible, I said as much, and then proposed a different approach. The interviewer persisted and claimed that it could be done with 'the outbox pattern'. I disagreed and ended the interview there. Later when I was chatting about it with a former colleague, he said "…

I'm a bit confused by the story. Why did you disagree?

Re: Anti-patterns in event-driven architecture

#84

Post by YZF »

I've worked in a large company where some variation of event driven architecture was used everywhere and treated as the word of G-d. Fairly successfully. Mostly in applications that ran on a single machine.

I've ended up in a lot of arguments about this while we were building larger distributed systems because I've come from a more request/response oriented message passing architectures. I.e. more synchronous. What I've found is that the event driven architecture did tend to lead to less abstractions and more leaked internal details. This isn't fundamental (you can treat events like an API) but was related to some details in our implementation (something along the line of CDC).

Another problem with distributed systems with persistent queues passing events is that if the consumer falls behind you start developing a lag. Yet another considerations is that the infrastructure to support this tends to have some performance penalties (e.g. going through Kafka with an event ends up being a lot more expensive than an RPC call). Overall it IMO makes for a lot of additional complexity which you may need in some cases, but if you don't then you shouldn't pay the cost.

What I've come to realize is that in many ways those systems are equivalent. You can simulate one over the other. If you have an event based system you can send requests as events and then wait for the response event. If you have a request/response system you can simulate events over that.

If we look at things like consensus protocols or distributed/persistent queues then obviously we would need some underlying resources (e.g. you might need a database behind your request/response model). So... Semantics. Don't know if others have a similar experience but when one system is mandated people will invent workarounds that end up looking like the other paradigm, which makes things worse.

There are things that conceptually fit well with an event driven architecture and then there are things that fit well with a request/response model. I'm guessing most large scale complex distributed apps would be best supporting both models.

Re: Anti-patterns in event-driven architecture

#85

Post by lolive »

marcosdumay wrote: post #49
jgraettinger1 wrote: post #21

Earlier quoted context omitted.

PostgreSQL. The WAL is an event log, and when you squint at its internal architecture, you’ll see plenty of overlap with distributed event sourcing.

Almost every modern software system. Anything running over the Web is event driven.

99.99% of the data we consume on the Web comes out of databases [call it Transactional-SQL-xxx or ColumnBased-yyy or Elastic-SaaS-zzz].

Re: Anti-patterns in event-driven architecture

#86

Post by YZF »

junto wrote: post #66
arwhatever wrote: post #13

I often hear the argument in favor of event-driven architecture that you can work on one part of a system in isolation without having to consider the other parts, and then I get assigned some task which requires me to consider the entire system operation, now with events that are harder to trace than function calls would have been. Now when people argue “because decoupling,” I hear, “You don’t get as much notificatio…

You need to improve your telemetry to feel the benefits. I can trace all the way through multiple services easily on a simple detailed flame graph in our systems. https://www.datadoghq.com/knowledge-center/distributed-traci... Unless you have a single monolith, you’re going to face issues with versioning whether it’s event based or API based. In each case you can usually add new properties to a message, but you can’t…

There are many examples of event sourcing but perhaps the most classical one is the bank account.

It's not just about auditing, it's also about transactionality and atomicity.

If you want to withdraw $5 from your account, the traditional approach of locking, updating everything, unlocking (or in other words wrapping everything in a transaction) doesn't scale as well as the notion that you just record the transaction (event). Implementation-wise this withdrawal can involve, updating two accounts and updating the audit/account transaction logs. We also want this to scale since our bank has millions of customers all operating more or less concurrently. A distributed log (like Kafka) is easy to scale and easy to reason about. You just insert the transaction record and you have a distributed system that will scale and is easy to reason about.

Another driver/flavour for something like event sourcing is what some might call state-based or state-oriented programming. That is instead of modifying state directly you are synchronizing state via events. This lets you e.g. code state machines around those that can lead, again, to easier to reason about (and test) code.

Re: Anti-patterns in event-driven architecture

#87

Post by NomDePlum »

junto wrote: post #66
arwhatever wrote: post #13

I often hear the argument in favor of event-driven architecture that you can work on one part of a system in isolation without having to consider the other parts, and then I get assigned some task which requires me to consider the entire system operation, now with events that are harder to trace than function calls would have been. Now when people argue “because decoupling,” I hear, “You don’t get as much notificatio…

You need to improve your telemetry to feel the benefits. I can trace all the way through multiple services easily on a simple detailed flame graph in our systems. https://www.datadoghq.com/knowledge-center/distributed-traci... Unless you have a single monolith, you’re going to face issues with versioning whether it’s event based or API based. In each case you can usually add new properties to a message, but you can’t…

Depends what domain you work in. Auditability is a key/mandatory requirement in a lot of regulated industries.

There are of course other ways to do auditability.

Event Sourcing + Projections provide a nice way to build multiple models/views from the same dataset. This can provide a lot of simplification for client code.

Re: Anti-patterns in event-driven architecture

#88

Post by thr0w »

candiddevmike wrote: post #3

Can someone share some long term event driven success stories? Almost everything you see online is written by consultants or brand new, greenfield implementations, curious how long these systems last.

> Can someone share some long term event driven success stories?

JavaScript

Re: Anti-patterns in event-driven architecture

#89

Post by lmm »

mrkeen wrote: post #71
arwhatever wrote: post #13

I often hear the argument in favor of event-driven architecture that you can work on one part of a system in isolation without having to consider the other parts, and then I get assigned some task which requires me to consider the entire system operation, now with events that are harder to trace than function calls would have been. Now when people argue “because decoupling,” I hear, “You don’t get as much notificatio…

> now with events that are harder to trace than function calls would have been I don't know how this could be true. Events are things - nouns which can be backed-up, replicated, stored, queried, rendered, indexed and searched over.

There's no "find usages" for events, and it becomes harder to find out why something didn't happen. A function call can't simply "not return" - in the worst case you get an exception, or a stuck thread in the caller that will show up in a stack dump. But downstream event processing can very easily just not happen, for one of many different reasons, and out-of-the-box it's often difficult to investigate.

Re: Anti-patterns in event-driven architecture

#90

Post by thr0w »

tkiolp4 wrote: post #35
candiddevmike wrote: post #3

Can someone share some long term event driven success stories? Almost everything you see online is written by consultants or brand new, greenfield implementations, curious how long these systems last.

No. The usual pains are: - Producer and consumer are decoupled. That’s a good thing m right? Good luck finding the consumer when you need to modify the producer (the payload). People usually don’t document these things - Let’s use SNS/SQS because why not. Good luck reproducing producers and consumers locally in your machine. Third party infra in local env is usually an afterthought - Observability. Of rather the lack…

> - Let’s use SNS/SQS because why not.

The amount of times I've come across someone who's inserted SQS into the mix to "speed things up"...

Post reply on HN