Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Dual Nature of Events in Event-Driven Architecture

reactivesystems.eu

Post reply on HN
Print view View original

21–30 of 66 posts

Re: The Dual Nature of Events in Event-Driven Architecture

#21

Post by gunnarmorling »

lutzh wrote: post #2

Article observing that events in event-driven architecture are both triggers of actions and carriers of data, and that these roles may conflict in the event design. Submitted by author.

Nice one. I wrote about this a while ago, from a slightly different perspective, focusing on data change events [1]. Making a similar differentiation there between id-only events (which you describe as triggers of action; from a data change feed perspective, that action typically would be a re-select of the current state of the represented record), full events (your carriers of data) and patch events (carriers of data with only the subset of attributes whose value has changed).

[1] https://www.decodable.co/blog/taxonomy-of-data-change-events

Re: The Dual Nature of Events in Event-Driven Architecture

#22

Post by lutzh »

kaba0 wrote: post #18

I am going on a bit of a tangent here, but I always wondered, are those of you who use absolutely huge event-driven architectures, have you ever got yourself into a loop? I can't help but worry about such, as event systems are fundamentally Turing-complete, and with a complex enough system it doesn't seem too hard to accidentally send an event because A, which will eventually, multiple other events later again cases…

Reg. the "technical" question: Kafka or any log-based message broker (or any message queue) would not prevent you from that. Any service can publish/send and/or subscribe/receive.

Regarding if it's a problem or a regular occurrence: No, really not. I have never seen this being a problem, I think that fear is unfounded.

Re: The Dual Nature of Events in Event-Driven Architecture

#23

Post by lutzh »

dkarl wrote: post #19

Events are published observations of facts. If you want to be able to use them as triggers, or to build state, then you have to choose the events and design the system to make that possible, but you always have to ensure that systems only publish facts that they have observed. Most issues I've seen with events are caused by giving events imprecise names, names that mean more or less than what the events attest to. Fo…

Thanks for your feedback! Very good point on the naming. fwiw the idea was if you buy a cinema ticket, you are usually presented with some sort of seating plan and select the seat (basically putting them into the shopping cart). So SeatSelected would be the equivalent of "ItemAdded" to the shopping cart in an e-commerce application I guess. Please don't over-interpret the example. There isn't even a definition what that booking aggregate contains. The point was really only to say that in order to differentiate the events, we don't necessarily need distinct types (which would result in multiple schemas on a topic), but can instead encode it in one type/schema. Think of it like mapping in ORM - instead of "table per subclass", you can use "table per class hierarchy".

Re: The Dual Nature of Events in Event-Driven Architecture

#24

Post by lutzh »

jwarden wrote: post #14
bob1029 wrote: post #3

The triggering of the action is a direct consequence of the information an event contains. Whether or not an action is triggered should not be the responsibility of the event. If you are writing events with the intention of having them invoke some specific actions, then you should prefer to invoke those things directly. You should be describing a space of things that have occurred, not commands to be carried out. By…

Another architecture might be that the service responsible for Seat Selection emits a `SeatSelected` event, and another service responsible for updating bookings emits a `BookingUpdated(Reason: SeatSelected)` "fat" event. Same for `PaymentReceived` and `TicketIssued`. Both events would "describe a space of things that occurred" as @bob1029 suggests. The seat selection process for an actual airline probably needs to b…

Thanks for your feedback. I realize I should have elaborated the example a bit more, it's too vague. So, as I wrote in some other reply as well, please don't over-interpret it. The point was only to say that in order to differentiate the events, we don't necessarily need distinct types (which would result in multiple schemas on a topic), but can instead encode it in one type/schema. Like mapping in ORM - instead of "table per subclass", you can use "table per class hierarchy".

Re: The Dual Nature of Events in Event-Driven Architecture

#25

Post by marcosdumay »

gunnarmorling wrote: post #20
marcosdumay wrote: post #10

Earlier quoted context omitted.

Your event data must not be mutable. That's kind of the first rule of any event-based system. It doesn't really matter the architecture, if you decide to name the things "event", everybody's head will break if you make them mutable. If you decide to add mutation there in some way, you will need to rewrite the event stream, replacing entire events.

It's not about mutability of events, but about mutating the underlying data itself. If the event only says "customer 123 has been updated", and a consumer of that event goes back to the source of the event to query the full state of that customer 123, it may have been updated again (or even deleted) since the event was emitted. Depending on the use case, this may or may not be a problem. If the consumer is only inter…

Making a wacky 2-steps announcement protocol doesn't change the nature of your events.

If the consumer goes to your database and asks "what's the data for customer 123 at event F52A?" it better always get back the same data or "that event doesn't exist, everything you know is wrong".

Re: The Dual Nature of Events in Event-Driven Architecture

#26

Post by singron »

withinboredom wrote: post #11
candiddevmike wrote: post #9

Earlier quoted context omitted.

There should be some law that says strictly serialized process should never be broken into discreet services. Distributed locks and transactions are hell.

The best way to avoid distributed locks and transactions is to manually do the work. For example, instead of doing a distributed lock on two accounts when transferring funds, you might do this (which is the same as a distributed transaction, without the lock): 1. take money from account A 2. if failed, put money back into account A 3. put money into account B 4. if failed, put money back into account A In other words…

The better version of this is sagas, which is a kind of a simplified distributed transaction. If you do this without actually using sagas, you can really mess this up.

E.g. you perform step 2, but fail to record it. When resuming from crash, you perform step 2 again. Now A has too much money in their account.

Re: The Dual Nature of Events in Event-Driven Architecture

#27

Post by gunnarmorling »

marcosdumay wrote: post #25
gunnarmorling wrote: post #20

Earlier quoted context omitted.

It's not about mutability of events, but about mutating the underlying data itself. If the event only says "customer 123 has been updated", and a consumer of that event goes back to the source of the event to query the full state of that customer 123, it may have been updated again (or even deleted) since the event was emitted. Depending on the use case, this may or may not be a problem. If the consumer is only inter…

Making a wacky 2-steps announcement protocol doesn't change the nature of your events. If the consumer goes to your database and asks "what's the data for customer 123 at event F52A?" it better always get back the same data or "that event doesn't exist, everything you know is wrong".

> ... at event F52A

Sure, if the database supports this sort of temporal query, then you're good with such id-only events. But that's not exactly the default for most databases / data models.

Re: The Dual Nature of Events in Event-Driven Architecture

#28

Post by marcosdumay »

gunnarmorling wrote: post #27
marcosdumay wrote: post #25

Earlier quoted context omitted.

Making a wacky 2-steps announcement protocol doesn't change the nature of your events. If the consumer goes to your database and asks "what's the data for customer 123 at event F52A?" it better always get back the same data or "that event doesn't exist, everything you know is wrong".

> ... at event F52A Sure, if the database supports this sort of temporal query, then you're good with such id-only events. But that's not exactly the default for most databases / data models.

I'm understanding what you have isn't really "events", but some kind of "notifications".

Events are part of a stream that define your data. The stream doesn't have to be complete, but if it doesn't make sense to do things like buffer or edit it, it's probably something else and using that name will mislead people.

Re: The Dual Nature of Events in Event-Driven Architecture

#29

Post by magicalhippo »

exabrial wrote: post #17

Well put! We do a lot of event driven architecture with ActiveMQ. We try to stick messaging-as-signalling rather than messaing-as-data-transfer. These are the terms we came up with, I'm sure Martin Fowler or someone else has described it better! So we have SystemA that is completing some processing of something. It's going to toss a message onto a queue that SystemB is listening on. We use an XA to make sure the data…

> We try to stick messaging-as-signalling rather than messaing-as-data-transfer.

I was thinking of trigger-messages vs data-messages.

Then again we've just begun dabbling with AMQP as part of transitioning our legacy application to a new platform, so a n00b in the field.

We do have what you might consider hybrid messages, where we store incoming data in an object store and send a trigger-message with the key to process the data. This keeps the queue lean and makes it easy for support to inspect submitted data after it's been processed, to determine if we have a bug or it's garbage in garbage out.

Re: The Dual Nature of Events in Event-Driven Architecture

#30

Post by nyrikki »

candiddevmike wrote: post #13
withinboredom wrote: post #11

Earlier quoted context omitted.

The best way to avoid distributed locks and transactions is to manually do the work. For example, instead of doing a distributed lock on two accounts when transferring funds, you might do this (which is the same as a distributed transaction, without the lock): 1. take money from account A 2. if failed, put money back into account A 3. put money into account B 4. if failed, put money back into account A In other words…

> This also requires that you have some kind of mechanism to handle an application crash between 2 and 3, but that is something else entirely Like a distributed transaction or lock. This is the entire problem space, your example above is very naive.

It is not a DTC, despite the DB world using ATM's wrongly as an example for decades, follow their model for actually moving money and you would be sent to jail.

"Accountants don't use erasers"

The ledger is the source of truth in accounting, if you use event streams as the source of truth you can gain the same advantages and disadvantages.

An event being past tense ONLY is a very critical part.

There are lots of way to address this all with their own tradeoffs and it is a case of the least worst option for a particuar context.

but over-reliance on ACID DBMSs and the false claim that ATMs use DTC really does hamper the ability to teach these concepts.

Post reply on HN