Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anti-patterns in event-driven architecture

codeopinion.com

Post reply on HN
Print view View original

101–110 of 171 posts

Re: Anti-patterns in event-driven architecture

#101

Post by YZF »

lmm wrote: post #99
YZF wrote: post #98

Earlier quoted context omitted.

Everything is a means and not an end but decoupling via an explicit API makes change easier. Spreading state across your system via events (specifically synchronizing data across systems via events relating to how that data changes) creates coupling. re: Huh. Sorry I was not clear there. What I meant is you can not create persistent queue semantics out of a request/response model without being able to make certain ki…

> decoupling via an explicit API makes change easier. Spreading state across your system via events (specifically synchronizing data across systems via events relating to how that data changes) creates coupling. An explicit API comes at a cost; the way I'd put it is that the inherently lower coupling of events (because e.g. you can publish the same events in multiple formats, whereas a request-response API generally…

Usually when you're ingesting something into Kafka it's important to know whether that was successful or not, hence the more or less inherent request/response that's part of that. That said it's an interesting thought experiment to see how far you can go without that.

When I think of large scale success stories around the request/response model I think AWS (where famously Bezos mandated APIs first) and Google. Both now have services that look more event oriented (e.g. SQS or Firebase). And ofcourse the modern web (though the ugly hacks needed to make something look like event driven was certainly not fun).

Events related to data changes are about keeping data structures in sync via events. Also known as state-based architecture. Something I worked on in the early 2000's kept a remote client/UI in sync with the server and a database using events like that and was a pretty lean/neat implementation.

Good one on the Chrome re-making requests when you're online for an active tab. That's certainly an interesting use case.

My intuition is that some things are very naturally events. Let's say a packet arriving into your computer. A mouse click. And some things are naturally a request-response. Let's say calculating the Cosine of an angle. You can replace x = sin(y) with an event, and an event that comes back, but that feels awkward as a human. Maybe not the best example...

It's another variation on the sync vs. async debates I guess. Coroutines or callbacks...

Re: Anti-patterns in event-driven architecture

#102

Post by jesse__ »

thr0w wrote: post #88
candiddevmike wrote: post #3

Can someone share some long term event driven success stories? Almost everything you see online is written by consultants or brand new, greenfield implementations, curious how long these systems last.

> Can someone share some long term event driven success stories? JavaScript

I think we have very different definitions of success

Re: Anti-patterns in event-driven architecture

#103

Post by OccamsMirror »

moandcompany wrote: post #26
cjk2 wrote: post #23

Earlier quoted context omitted.

I call this one FDD: Fuckwit Driven Development. Because if it was resume-driven I'd expect it to be something that they would want to put on their resume. But this is unmentionable.

There's sayings along the lines of "Victory/success has a thousand fathers, but defeat/failure is an orphan." Chances are that system, and its outcomes are described very differently on a resume

As long as the list of technologies used is impressive sounding you're on to a winner.

Re: Anti-patterns in event-driven architecture

#104

Post by RHSman2 »

smrtinsert wrote: post #57
pudwallabee wrote: post #22

I have seen Kafka pulled out by its hairs and replaced with request based architecture. Event driven architecture, to me is itself an antipattern. It seems like a replacement for batch processing. Replayable messages are AWESOME. Until you encounter the complexity for a system to actually replay them consistently. As far as the authors video, while there was some truth in there, it was a little thin, compared to the…

> and the n distributions of Kafka logs in your organization could be 1000x more expensive than a monolithic DB and a monolithic API to maintain Not to mention certain observability vendors bleeding you for all those logs you now need to keep an eye on it. Absolutely agreed on every point

The unseen critical part of the equation

Re: Anti-patterns in event-driven architecture

#105

Post by majormajor »

trevor-e wrote: post #47
arwhatever wrote: post #13

I often hear the argument in favor of event-driven architecture that you can work on one part of a system in isolation without having to consider the other parts, and then I get assigned some task which requires me to consider the entire system operation, now with events that are harder to trace than function calls would have been. Now when people argue “because decoupling,” I hear, “You don’t get as much notificatio…

Like anything it can be abused and sometimes folks go overboard with turning everything into an event. However, when done right, it is really amazing to work with. As an event producer as long as you follow reasonable backwards-compatibility best practices then you should be pretty safe from breaking things downstream. As a consumer, follow defensive programming and allow for idempotency in case you need to reprocess…

> As an event producer as long as you follow reasonable backwards-compatibility best practices then you should be pretty safe from breaking things downstream.

That can protect you from "downstream can't even read the message anymore" but it doesn't help you with the much more common "downstream isn't doing the right thing with the message anymore" problem. Schema evolution is kinda like schema'd RPC calls vs plain JSON: it will protect you from "oops, we sent eventId instead of event_id" type of errors, but won't prevent you from making logical errors. In a larger org, this can turn into delayed-discovery nightmares.

A synchronous API call could give you back an error response and alert your immediately to something being wrong. The system notifies you directly.

A downstream event consumer may fail in ways entirely off of your team's radar. The downstream team starts getting alerts. Whether or not those alerts make it immediately obvious to them that it's your fault... that depends on a bunch of factors.

Re: Anti-patterns in event-driven architecture

#106

Post by chenster »

blowski wrote: post #10
candiddevmike wrote: post #3

Can someone share some long term event driven success stories? Almost everything you see online is written by consultants or brand new, greenfield implementations, curious how long these systems last.

I was the lead developer on one for an insurance company a few years back, and it’s still in active use. Insurance is a heavily regulated domain, where an audit trail is more important than performance. There was a natural pattern for it to follow, as we were mapping a stable industry standard. I also tried doing it in a property setting, where profit margins were tight. The effort needed wasn’t worth the cost, and c…

What did you mean traditional crud as oppose to event-driven arch? How is it relevant to the subject in dicussion?

Re: Anti-patterns in event-driven architecture

#107

Post by chenster »

rswail wrote: post #63
candiddevmike wrote: post #3

Can someone share some long term event driven success stories? Almost everything you see online is written by consultants or brand new, greenfield implementations, curious how long these systems last.

Wrote a public transport ticketing system that processes 100-200K+ trips/day with sub-second push of notification to mobiles of trip/payments. Event driven and CQRS "entities" made logic and processing much easier to create/test/debug. Primary issues: 1. Making sure you focus on the "Nouns" (entities) not the "Verbs". 2. Kafka requiring polling for consumers sucks if you want to "scale to zero". 3. Sharding of event…

Can you elaborate #1 Nouns over Vers?

Re: Anti-patterns in event-driven architecture

#108

Post by majormajor »

pudwallabee wrote: post #22

I have seen Kafka pulled out by its hairs and replaced with request based architecture. Event driven architecture, to me is itself an antipattern. It seems like a replacement for batch processing. Replayable messages are AWESOME. Until you encounter the complexity for a system to actually replay them consistently. As far as the authors video, while there was some truth in there, it was a little thin, compared to the…

I haven't run into weird Kafka data loss issues like you describe - although, I will note, a lot of applications don't actually have much testing to notice something like 1 in 10k messages being dropped if it was happening.[0]

But when I've done that testing, Kafka hasn't been the problem.

The problem I've run into most is that ordering is a giant fucking pain in the ass if you actually want consistent replayability and don't have trivial partitioning needs. Some consumers want things in order by customer ID, other consumers want things in order by sold product ID, others by invoice ID? Uh oh. If you're thinking you could easily replay to debug, the size and scope of the data you have to process for some of those cases just exploded. Or you wrote N times, once for each of those, and then hopefully your multi-write transaction implementation was perfect!

[0] in fairness, a lot of applications also don't guarantee that they never drop requests at all, obviously. 500 and retry and hope that you don't run out of retries very often; if you do, it's just dropped on the ground and it's considered acceptable loss to have some of that for most companies/applications.

Re: Anti-patterns in event-driven architecture

#109

Post by soulofmischief »

lmm wrote: post #96
soulofmischief wrote: post #92

Earlier quoted context omitted.

In JavaScript, const myEvent = 'myEvent', target = new EventTarget() target.on( myEvent, () => { console.log( "It's easy to introspect well-organized code." ) }) target.dispatchEvent( new Event( myEvent ))

Yeah, good luck remembering to do that up-front for every event handler. You missed one? Whoops, enjoy the information you wanted silently not being there when you need it.

Remembering to do what? Properly maintain a list of constants and enums to use throughout my application?

That's not something I have to remember or forget, it's a simple habit that is as natural as importing and referencing a function.

As a general rule, numbers and string literals should never be hardcoded. Internalizing this should be a base expectation of any high-performing team member.

Re: Anti-patterns in event-driven architecture

#110

Post by [deleted] »

majormajor wrote: post #108
pudwallabee wrote: post #22

I have seen Kafka pulled out by its hairs and replaced with request based architecture. Event driven architecture, to me is itself an antipattern. It seems like a replacement for batch processing. Replayable messages are AWESOME. Until you encounter the complexity for a system to actually replay them consistently. As far as the authors video, while there was some truth in there, it was a little thin, compared to the…

I haven't run into weird Kafka data loss issues like you describe - although, I will note, a lot of applications don't actually have much testing to notice something like 1 in 10k messages being dropped if it was happening.[0] But when I've done that testing, Kafka hasn't been the problem. The problem I've run into most is that ordering is a giant fucking pain in the ass if you actually want consistent replayability…

[deleted]
Post reply on HN