Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anti-patterns in event-driven architecture

codeopinion.com

Post reply on HN
Print view View original

131–140 of 171 posts

Re: Anti-patterns in event-driven architecture

#131

Post by roncesvalles »

onetimeuse92304 wrote: post #130

Not specifically about event-driven, but the most damaging anti-pattern I would say is microservices. In pretty much all projects I worked with in recent years, people chop up the functionality into small separate services and have the events be serialised, sent over the network and deserialised on the other side. This typically causes enormous waste of efficiency and consequently causes applications to be much more…

Also, the single most nonsensical reason that people give for doing microservices is that "it allows you to scale parts of the application separately". Why the fuck do you need to do that? Do you scale every API endpoint separately based on the load that it gets? No, of course not. You scale until the hot parts have manageable load and the cold parts will just tag along at no cost. The only time this argument makes sense is if one part is a stateless application and the other part is a database or cache cluster.

Microservices make sense when there are very strong organizational boundaries between the parts (you'd have to reinterview to move from one team to the other), or if there are technical reasons why two parts of the code cannot share the same runtime environment (such as being written in different languages), and a few other less common reasons.

Re: Anti-patterns in event-driven architecture

#132

Post by onetimeuse92304 »

roncesvalles wrote: post #131
onetimeuse92304 wrote: post #130

Not specifically about event-driven, but the most damaging anti-pattern I would say is microservices. In pretty much all projects I worked with in recent years, people chop up the functionality into small separate services and have the events be serialised, sent over the network and deserialised on the other side. This typically causes enormous waste of efficiency and consequently causes applications to be much more…

Also, the single most nonsensical reason that people give for doing microservices is that "it allows you to scale parts of the application separately". Why the fuck do you need to do that? Do you scale every API endpoint separately based on the load that it gets? No, of course not. You scale until the hot parts have manageable load and the cold parts will just tag along at no cost. The only time this argument makes s…

Oh, it is even worse.

The MAIN reason for microservices was that you could have multiple teams work on their services independently from each other. Because coordinating work of multiple teams on a single huge monolithic application is a very complex problem and has a lot of overhead.

But, in many companies the development of microservices/agile teams is actually synchronised between multiple teams. They would typically have common release schedule, want to deliver larger features across multitude of services all at the same time, etc.

Effectively making the task way more complex than it would be with a monolithic application

Re: Anti-patterns in event-driven architecture

#133

Post by pyrale »

mattacular wrote: post #127
pyrale wrote: post #119

Earlier quoted context omitted.

Data consumers in unknown teams is a nightmare regardless of the architecture. Events sent for readership you can’t control are ideally of the type «x changed», and the consumer must then fetch data on the relevant endpoint. That or the company must have serious versioning policies.

What's the benefit of using an asynchronous event driven system if you can't process any of those events without performing a synchronous query back on the same provider for all of the necessary data?

You get relevant notifications without polling or needing to sub, and you don’t have to be strict about message versioning.

Re: Anti-patterns in event-driven architecture

#134

Post by lomase »

YZF wrote: post #86
junto wrote: post #66

Earlier quoted context omitted.

You need to improve your telemetry to feel the benefits. I can trace all the way through multiple services easily on a simple detailed flame graph in our systems. https://www.datadoghq.com/knowledge-center/distributed-traci... Unless you have a single monolith, you’re going to face issues with versioning whether it’s event based or API based. In each case you can usually add new properties to a message, but you can’t…

There are many examples of event sourcing but perhaps the most classical one is the bank account. It's not just about auditing, it's also about transactionality and atomicity. If you want to withdraw $5 from your account, the traditional approach of locking, updating everything, unlocking (or in other words wrapping everything in a transaction) doesn't scale as well as the notion that you just record the transaction…

As far as I know banks in my country use transactions and IBM mainframes, not distributed systems.

Re: Anti-patterns in event-driven architecture

#135

Post by withinboredom »

onetimeuse92304 wrote: post #132
roncesvalles wrote: post #131

Earlier quoted context omitted.

Also, the single most nonsensical reason that people give for doing microservices is that "it allows you to scale parts of the application separately". Why the fuck do you need to do that? Do you scale every API endpoint separately based on the load that it gets? No, of course not. You scale until the hot parts have manageable load and the cold parts will just tag along at no cost. The only time this argument makes s…

Oh, it is even worse. The MAIN reason for microservices was that you could have multiple teams work on their services independently from each other. Because coordinating work of multiple teams on a single huge monolithic application is a very complex problem and has a lot of overhead. But, in many companies the development of microservices/agile teams is actually synchronised between multiple teams. They would typica…

I've worked with thousands of other employees on a single monolithic codebase, which was delivered continuously. There was no complex overhead.

The process went something like this:

1. write code

2. get code review from my team (and/or the team whose code I was touching)

3. address feedback

4. on sign-off, merge and release code to production

5. monitor logs/alerts for increase in errors

In reality, even with thousands of developers, you don't have thousands of merges per day, it was more like 30-50 PRs being merged per day and on a multi-million line codebase, most PR's were never anywhere near each other.

Re: Anti-patterns in event-driven architecture

#136

Post by hectormalot »

lomase wrote: post #134
YZF wrote: post #86

Earlier quoted context omitted.

There are many examples of event sourcing but perhaps the most classical one is the bank account. It's not just about auditing, it's also about transactionality and atomicity. If you want to withdraw $5 from your account, the traditional approach of locking, updating everything, unlocking (or in other words wrapping everything in a transaction) doesn't scale as well as the notion that you just record the transaction…

As far as I know banks in my country use transactions and IBM mainframes, not distributed systems.

I once spoke with some engineers at Monzo and at the time (2018) financial transactions were handled via a combination of Kafka and Cassandra.

Re: Anti-patterns in event-driven architecture

#137

Post by withinboredom »

majormajor wrote: post #108
pudwallabee wrote: post #22

I have seen Kafka pulled out by its hairs and replaced with request based architecture. Event driven architecture, to me is itself an antipattern. It seems like a replacement for batch processing. Replayable messages are AWESOME. Until you encounter the complexity for a system to actually replay them consistently. As far as the authors video, while there was some truth in there, it was a little thin, compared to the…

I haven't run into weird Kafka data loss issues like you describe - although, I will note, a lot of applications don't actually have much testing to notice something like 1 in 10k messages being dropped if it was happening.[0] But when I've done that testing, Kafka hasn't been the problem. The problem I've run into most is that ordering is a giant fucking pain in the ass if you actually want consistent replayability…

I would say that requiring in-order events is a huge anti-pattern. What guarantee do you have that they were actually produced in-order and all the clocks are in-sync enough to know that without a doubt?

Re: Anti-patterns in event-driven architecture

#138

Post by sharlos201068 »

zdragnar wrote: post #81
Osiris wrote: post #58

Earlier quoted context omitted.

What I like about event driven is that you don't even need to know if anyone is listening to or cares about your event. And as a consumer, many independent tasks can be triggered by the same event. I'm working on a system right now and because of events, it's very easy for me to write a handler for when a certain type of record is created in the database. My feature depends on knowing that new record was made so we c…

> you don't even need to know if anyone is listening to or cares about your event. Right up until you need to change something about the event because the business logic it represents has changed. Then you suddenly need to track down all the systems that have been relying on it, including that one that nobody knows anything about and always forgets exists because some guy decided to implement the service in erlang an…

How is that any different for an API-driven architecture? You'd need to track down all consumers of your API you're wanting to make a breaking change to.

Re: Anti-patterns in event-driven architecture

#139

Post by sharlos201068 »

ninkendo wrote: post #28
candiddevmike wrote: post #3

Can someone share some long term event driven success stories? Almost everything you see online is written by consultants or brand new, greenfield implementations, curious how long these systems last.

Chiming in with another “no” here. We adopted a message bus/event-driven architecture when moving a very popular piece of software from the cloud, to directly on the user’s device… it was a disaster IMO. The core orchestration of the system was done via events on the bus, and nobody had any idea what was happening when a bug occurred. People would pass bugs around, “my code did the right thing given the event it got”…

We use an event driven architecture at work and find it works quite well, however events are for communicating between services across business domains and owned by different teams.

If you have some logic A and B running on user input, I wouldn't be splitting that across different services.

Re: Anti-patterns in event-driven architecture

#140

Post by nithril »

okr wrote: post #69
junto wrote: post #66

Earlier quoted context omitted.

You need to improve your telemetry to feel the benefits. I can trace all the way through multiple services easily on a simple detailed flame graph in our systems. https://www.datadoghq.com/knowledge-center/distributed-traci... Unless you have a single monolith, you’re going to face issues with versioning whether it’s event based or API based. In each case you can usually add new properties to a message, but you can’t…

A niche requirement? There are big accounting firms who organize payrolls, where the changes that you mentioned are an important part of their business. There are also other companies, which do the typical snapshot and roll up to the current time, when they start the services, that need the data without having access to the database.

You don't need event sourcing to organize payrolls event "at scale".
Post reply on HN