Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anti-patterns in event-driven architecture

codeopinion.com

Post reply on HN
Print view View original

151–160 of 171 posts

Re: Anti-patterns in event-driven architecture

#151

Post by onetimeuse92304 »

withinboredom wrote: post #135
onetimeuse92304 wrote: post #132

Earlier quoted context omitted.

Oh, it is even worse. The MAIN reason for microservices was that you could have multiple teams work on their services independently from each other. Because coordinating work of multiple teams on a single huge monolithic application is a very complex problem and has a lot of overhead. But, in many companies the development of microservices/agile teams is actually synchronised between multiple teams. They would typica…

I've worked with thousands of other employees on a single monolithic codebase, which was delivered continuously. There was no complex overhead. The process went something like this: 1. write code 2. get code review from my team (and/or the team whose code I was touching) 3. address feedback 4. on sign-off, merge and release code to production 5. monitor logs/alerts for increase in errors In reality, even with thousan…

50 PRs with a thousand developers is definitely not healthy situation.

It means any developer merges their work very, very rarely (20 days = 4 weeks on average...) and that in my experience means either low productivity (they just produce little) or huge PRs that have lots of conflicts and are PITA to review.

Re: Anti-patterns in event-driven architecture

#152

Post by withinboredom »

onetimeuse92304 wrote: post #151
withinboredom wrote: post #135

Earlier quoted context omitted.

I've worked with thousands of other employees on a single monolithic codebase, which was delivered continuously. There was no complex overhead. The process went something like this: 1. write code 2. get code review from my team (and/or the team whose code I was touching) 3. address feedback 4. on sign-off, merge and release code to production 5. monitor logs/alerts for increase in errors In reality, even with thousan…

50 PRs with a thousand developers is definitely not healthy situation. It means any developer merges their work very, very rarely (20 days = 4 weeks on average...) and that in my experience means either low productivity (they just produce little) or huge PRs that have lots of conflicts and are PITA to review.

Heh, PRs were actually quite small (from what I saw), and many teams worked on their own repos and then grafted them into the main repo (via subtrees and automated commits). My team worked in the main repo, mostly on framework-ish code. I also remember quite a bit of other automated commits as well (mostly built caches for things that needed to be served sub-ms but changed very infrequently).

And yes, spending two-to-three weeks on getting 200 lines of code absolutely soul-crushingly perfect, sounds about right for that place but that has nothing to do with it being a monolith.

Re: Anti-patterns in event-driven architecture

#153

Post by sophiabits »

onetimeuse92304 wrote: post #149
SadCordDrone wrote: post #146

Earlier quoted context omitted.

Also - you give up type safety and refactoring. LoL

Well, technically, you can construct the microservices preserving type safety. You can have an interface with two implementations - on the service provider, the implementation provides the actual functionality, - on the client, the implementation of the interface is just a stub connecting to the actual service provider. Thus you can sort of provide separation of services as an implementation detail. However in practi…

Even with this setup in place you need a heightened level of caution relative to a monolith. In a monolith I can refactor function signatures however I desire because the whole service is an atomically deployed unit. Once you have two independently deployed components that goes out the window and you now need to be a lot more mindful when introducing breaking changes to an endpoint’s types

Re: Anti-patterns in event-driven architecture

#154

Post by Pet_Ant »

SadCordDrone wrote: post #146
onetimeuse92304 wrote: post #130

Not specifically about event-driven, but the most damaging anti-pattern I would say is microservices. In pretty much all projects I worked with in recent years, people chop up the functionality into small separate services and have the events be serialised, sent over the network and deserialised on the other side. This typically causes enormous waste of efficiency and consequently causes applications to be much more…

Also - you give up type safety and refactoring. LoL

You don't have to. The producers of the microservice also produces an adapter. The adapter looks like a regular local service, but it implements the code as a REST request to another microservice. This was you get you type-safety. Generally you structure the code as

Proj:

|-proj-api

|-proj-client

|-proj-service

Both proj-client and proj-service consume/depend-on proj-api so they are in sync of what is going on.

Now, you can switch the implementation of the service to gRPC if you wanted with full source compatibility. Or move it locally.

Re: Anti-patterns in event-driven architecture

#155

Post by lmm »

friendzis wrote: post #121
lmm wrote: post #89

Earlier quoted context omitted.

There's no "find usages" for events, and it becomes harder to find out why something didn't happen. A function call can't simply "not return" - in the worst case you get an exception, or a stuck thread in the caller that will show up in a stack dump. But downstream event processing can very easily just not happen, for one of many different reasons, and out-of-the-box it's often difficult to investigate.

> A function call can't simply "not return" Remember "callback hell"? Assumption that a function call returns after running to completion requires rather specific synchronous cascading architecture, which WILL break in multithreaded code. Most of the multithreaded function calls will set a flag in shared memory and return early, expecting caller to poll. If your API is based on single entry-point `invokeMethod(callee…

> Most of the multithreaded function calls will set a flag in shared memory and return early, expecting caller to poll.

Which is exactly switching from function calls to event-driven architecture, and the problems with that are exactly the problems we're talking about.

Re: Anti-patterns in event-driven architecture

#156

Post by YZF »

bux93 wrote: post #129
YZF wrote: post #84

I've worked in a large company where some variation of event driven architecture was used everywhere and treated as the word of G-d. Fairly successfully. Mostly in applications that ran on a single machine. I've ended up in a lot of arguments about this while we were building larger distributed systems because I've come from a more request/response oriented message passing architectures. I.e. more synchronous. What I…

Every synchronous call is, in fact, asynchronous. We just hide it on the stack, in the return address, in the TCP connection etc. No call is really blocking anymore, the OS or the CPU will run some other thread. People who insist that things are simpler in a synchronous model are just ignoring the actual mechanics. Which is fine, that's just abstraction.

Well, a CALL instruction on your CPU is pretty much synchronous. But it's true that sometimes what looks synchronous is often asynchronous under the hood. The CPU itself is I guess "asynchronous", a bunch of flipflops and signals ("events") ;)

I can recall software where I tried to wrestle a bunch of asynchronous things into looking more synchronous and then software where I really enjoyed working with a pure asynchronous model (Boost.Asio FTW). Usually the software where I want things to be synchronous is where for the most part I want to execute a linear sequence of things that depend on each other without really being able to use that time for doing other things vs. software where I want all things to happen at the same time all the time (e.g. being able to take in new connections over the network, serve existing connections etc.) and spinning threads for doing that is not a good fit (performance or abstraction-wise).

The locality of the synchronous model makes it easier to grok as long as you're ok with not being able to do something else while the asynchronous thing is going on. OTOH state machines, or statecharts to go further, which are an inherently asynchronous view, have many advantages (But are not Turing Complete).

Re: Anti-patterns in event-driven architecture

#157

Post by friendzis »

lmm wrote: post #155
friendzis wrote: post #121

Earlier quoted context omitted.

> A function call can't simply "not return" Remember "callback hell"? Assumption that a function call returns after running to completion requires rather specific synchronous cascading architecture, which WILL break in multithreaded code. Most of the multithreaded function calls will set a flag in shared memory and return early, expecting caller to poll. If your API is based on single entry-point `invokeMethod(callee…

> Most of the multithreaded function calls will set a flag in shared memory and return early, expecting caller to poll. Which is exactly switching from function calls to event-driven architecture, and the problems with that are exactly the problems we're talking about.

The problems you describe are inherent to indirect invocations and are related to event-driven architectures only because typical event dispatching architecture is built on non-blocking calls.

You do not even need return-early (non-blocking) semantics for these problems to manifest. You can implement a giant string-keyed vtable for all methods in your program (or use a language with advanced reflection capabilities) and will have exactly the same problems. Namely there probably won't be tooling to match caller-callee pairs, which is the core issue here.

Re: Anti-patterns in event-driven architecture

#158

Post by davedx »

YZF wrote: post #113
davedx wrote: post #112

Earlier quoted context omitted.

Do you know for a fact that banks use event sourcing for transactions? I thought they were extremely eventually consistent

I'm pretty sure different banks use vastly different architectures. Some run nightly batch jobs on mainframes that are written in COBOL. But alas, I've not worked for any bank, this is just a commonly used example. I am willing to bet the transaction log, or ledger, is indeed a very common approach since that's also common in accounting. EDIT: Also event sourcing would typically be eventually consistent. I imagine fo…

I’m sceptical, because I can’t find any examples of banks using it in production, just lots of blog posts by consultants and companies selling event sourcing solutions.

I did work somewhere that used a Kafka stack in production. It wasn’t a compelling use case and they spent almost an entire year on infra and productionizing it. It left me extremely sceptical about anything “big data streams” related :)

Re: Anti-patterns in event-driven architecture

#159

Post by mrkeen »

lastofus wrote: post #74
mrkeen wrote: post #72

Earlier quoted context omitted.

I had an interview where I was asked how I would guarantee that an event happened in addition to a database update (transactionally). It sounded kind of impossible, I said as much, and then proposed a different approach. The interviewer persisted and claimed that it could be done with 'the outbox pattern'. I disagreed and ended the interview there. Later when I was chatting about it with a former colleague, he said "…

I don't quite see what the outbox pattern has to do with the two generals problem. The point of the outbox pattern is that a durable record of the need to send an event is stored in the DB as part of the DB txn, taking advantage of ACID guarantees. Once you have that durable record in your DB, you can essentially treat your DB as a queue (there's lot of great articles on how to do this with Postgres for instance) for…

> I don't quite see what the outbox pattern has to do with the two generals problem.

Well then, hopefully you would have found it an unsatisfactory 'solution' and walked away from that interview too ;)

> Once you have that durable record in your DB, you can essentially treat your DB as a queue (there's lot of great articles on how to do this with Postgres for instance) for some worker processes to later process the queue records, and send the events.

Yeah but I already have a queue I can treat as a queue. It's called Kafka.

Re: Anti-patterns in event-driven architecture

#160

Post by mrkeen »

ramchip wrote: post #83
mrkeen wrote: post #72

Earlier quoted context omitted.

I had an interview where I was asked how I would guarantee that an event happened in addition to a database update (transactionally). It sounded kind of impossible, I said as much, and then proposed a different approach. The interviewer persisted and claimed that it could be done with 'the outbox pattern'. I disagreed and ended the interview there. Later when I was chatting about it with a former colleague, he said "…

I'm a bit confused by the story. Why did you disagree?

They asked how I would guarantee that a Postgres update and a Kafka event could both happen transactionally.

  (P, K)
Which sounds like one of those classical impossibility proofs.

Their solution was to introduce another part into the system, "the outbox":

  (P, O) K
P and O can form a transaction, but that still leaves the original problem unanswered, how to include K in the transaction.
Post reply on HN