Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Events: Fat or Thin?

codesimple.blog

Post reply on HN
Print view View original

11–20 of 61 posts

Re: Events: Fat or Thin?

#11

Post by SideburnsOfDoom »

doctor_eval wrote: post #8
SideburnsOfDoom wrote: post #4

Earlier quoted context omitted.

> thin events reduce coupling. Sure, the receiver might call an API and that creates coupling with the API. You make a statement in the first sentence, and in the next sentence produce evidence ... that the statement is wrong. And, YMMV. It is my experience that thin events add coupling. If service B receives an event, and wants to process it ASAP (i.e. near real time) and so calls back over http to Service A for the…

> You make a statement in the first sentence, and in the next sentence produce evidence ... that the statement is wrong. My first sentence was quoting from the article, then I refute the article. Sorry if that wasn’t clear. Re your point a), yes I agree in this case you’d send the contents in the body, but then I’d tend to call it stream processing rather than event processing - I admit this might seem like splitting…

I think it's a straw man to say "we couldn't eliminate all API calls, so fat events are useless" - even removing 1 dependency at a time is a win. In my experience, you generally can do this, and that was the approach taken for reliability improvement.

> it’s very difficult to efficiently create event bodies that contain everything that all receivers are going to need.

"everything that all receivers need" seems like another straw man, a "you won't get it perfect so don't try to improve". I've seen it work well enough to be worthwhile.

> From a maintenance perspective, the sender doesn’t know what the receivers depend on

At a glance, no. But it's not imponderable, assuming a limited number of in-house consumers. The absolute statement about it isn't accurate.

> it’s just an assumption that the receiver needs the version of data in the message, rather than the latest version. So I don’t think this is a strong argument for fat events.

I've seen it cause a severe and hard-to-diagnose failure, when system A lags enough, so I think it is a strong argument.

> Maybe I should have said, “I’m on team thin by default.

Sure. I'm on team "fat events" by default because it can solve more issues than it creates. If it turns out that 90% of the event gets ignored, with no issues or http call-backs, then this might be a case for thin events.

Re: Events: Fat or Thin?

#12

Post by SideburnsOfDoom »

delusional wrote: post #10
SideburnsOfDoom wrote: post #4

Earlier quoted context omitted.

> thin events reduce coupling. Sure, the receiver might call an API and that creates coupling with the API. You make a statement in the first sentence, and in the next sentence produce evidence ... that the statement is wrong. And, YMMV. It is my experience that thin events add coupling. If service B receives an event, and wants to process it ASAP (i.e. near real time) and so calls back over http to Service A for the…

b) You're asking for occasional "eventual consistency" trouble when A's state lags or has moved on ahead of the event If you allow A's state to lag behind it's own events, then how are you ever going to create a sane system? Surely A either has to be ahead or at the state that caused the event to emit, or events are pointless.

> A's state to lag behind it's own events,

Real systems don't have just 2 services. There can be 100s and the "own events" assumption may not hold.

Re: Events: Fat or Thin?

#13

Post by ivan_gammel »

I‘d pick neither and just let the system in possession of data send with the event only the part of data it owns (i.e. something in between fat and thin). Saves API call back, the body doesn’t have to be fully deserialized, so no format coupling, the rest can be fetched from other services on demand (coherent state though is not guaranteed, but that’s usually not critical with well designed bounded contexts).

Re: Events: Fat or Thin?

#14

Post by PaulRobinson »

FeepingCreature wrote: post #7
SideburnsOfDoom wrote: post #4

Earlier quoted context omitted.

> thin events reduce coupling. Sure, the receiver might call an API and that creates coupling with the API. You make a statement in the first sentence, and in the next sentence produce evidence ... that the statement is wrong. And, YMMV. It is my experience that thin events add coupling. If service B receives an event, and wants to process it ASAP (i.e. near real time) and so calls back over http to Service A for the…

> b) You're asking for occasional "eventual consistency" trouble when A's state lags or has moved on ahead of the event To be noted that this is the default if B is recovering after an outage. Personally, I consider events to be insane. "We create an immutable database so that the state of the system is always recoverable." Okay, cool, very functional programming of you. "But then to actually work with the event from…

The bit you seem to be missing is the events are the source of truth, not the databases.

Lose your database? Roll up all the events. Got a lot of them? Take snapshots and then roll up from the last trusted snapshot.

In true event sourced systems, the databases and stateful systems are artefacts that can be thrown away and rebuilt. The event log is the actual “true” database.

Once you design around that, your objections melt away.

And if you think this is some faddish trend, this is how finance has worked since the invention of book keeping and how your databases under your stateful services are working under the hood.

Re: Events: Fat or Thin?

#15

Post by mabbo »

As always, it depends. Yay engineering and trade-offs.

Hey just remember: both is always an option if you're consumers disagree. Thin stream from the consumers who don't trust the fat data, fat stream for the event log and other consumers that prefer it.

Re: Events: Fat or Thin?

#16

Post by doctor_eval »

SideburnsOfDoom wrote: post #11
doctor_eval wrote: post #8

Earlier quoted context omitted.

> You make a statement in the first sentence, and in the next sentence produce evidence ... that the statement is wrong. My first sentence was quoting from the article, then I refute the article. Sorry if that wasn’t clear. Re your point a), yes I agree in this case you’d send the contents in the body, but then I’d tend to call it stream processing rather than event processing - I admit this might seem like splitting…

I think it's a straw man to say "we couldn't eliminate all API calls, so fat events are useless" - even removing 1 dependency at a time is a win. In my experience, you generally can do this, and that was the approach taken for reliability improvement. > it’s very difficult to efficiently create event bodies that contain everything that all receivers are going to need. "everything that all receivers need" seems like a…

> I think it's a straw man to say "we couldn't eliminate all API calls, so fat events are useless"

Well, yes it is a straw man, because I never said that.

> At a glance, no. But it's not imponderable, assuming a limited number of in-house consumers. The absolute statement about it isn't accurate.

That’s a pretty huge assumption. Especially when one of the advantages of pub/sub is supposed to be decoupling.

Anyway, we clearly have had different experiences, and there is no silver bullet.

Re: Events: Fat or Thin?

#17

Post by delusional »

SideburnsOfDoom wrote: post #12
delusional wrote: post #10

Earlier quoted context omitted.

b) You're asking for occasional "eventual consistency" trouble when A's state lags or has moved on ahead of the event If you allow A's state to lag behind it's own events, then how are you ever going to create a sane system? Surely A either has to be ahead or at the state that caused the event to emit, or events are pointless.

> A's state to lag behind it's own events, Real systems don't have just 2 services. There can be 100s and the "own events" assumption may not hold.

Sure, but in a thin events model someone would "own" the events since otherwise the subscriber wouldn't know where to query the actual data. What would you even do with an event saying a customer changes address if querying that address then produces the old one.

I'm genuinely curious how such an architecture would work. You don't have to respond directly here, but if you have any reference to further reading, I'd appreciate it.

Re: Events: Fat or Thin?

#18

Post by doctor_eval »

avereveard wrote: post #6
doctor_eval wrote: post #3

> Thin Events add coupling That’s not my experience. In fact I’d say fat events add coupling because they create an invisible (from the emitter) dependency on the event body, which becomes ossified. So I’d say the opposite: thin events reduce coupling. Sure, the receiver might call an API and that creates coupling with the API. But receivers are also free to call or not call any other API they want. What if they don’…

assume the data format changes, it would change in the called api as well. as long as the fat event sends data that it's in the same format that the api would return, you'd have the same level of coupling. I think fat vs thin is more about how much other services the event have to travel, because thin event would multiply reads by a fair factor, with the tradeoff being the performance hit for the queue system to stor…

With an API you can publish a new endpoint (/v1, /v2 etc). It’s normally reasonably easy to maintain an old API even while you add features to the new API, and the runtime penalty is minimal because clients would be expected to call just one version of the API for any given event. (You can also see who’s calling the old API and ask them to change)

But this is not true for events. If you change the body such that you now need to maintain two versions of an event, then you have to publish both events simultaneously, which means double the server side effort, storage etc for each event version. It’s pretty inefficient, and painful. You can work out who subscribes to the old event but there is still a big efficiency hit.

You might be right about many reads per event in a simplistic way; if you have a lot of clients then it could be expensive if you don’t have a server side cache. But there would typically be a lot of temporality in such a system so it seems like an easy problem to solve for most use cases; you don’t have to cache for long, but caches are of course tricky if your use case is not very simple. That said, if there is already a HTTP connection open then the additional latency and bandwidth hit cause by this events are going to be minimal in most cases, and probably drowned out entirely if you need to push multiple versions.

As I said in another thread, I should have said that thin is my default. There are cases when fat makes more sense, but normally I’d start with thin and see if I need to flesh it out. Whenever I’ve started fat I’ve ended up reverting.

Re: Events: Fat or Thin?

#20

Post by manfre »

doctor_eval wrote: post #18
avereveard wrote: post #6

Earlier quoted context omitted.

assume the data format changes, it would change in the called api as well. as long as the fat event sends data that it's in the same format that the api would return, you'd have the same level of coupling. I think fat vs thin is more about how much other services the event have to travel, because thin event would multiply reads by a fair factor, with the tradeoff being the performance hit for the queue system to stor…

With an API you can publish a new endpoint (/v1, /v2 etc). It’s normally reasonably easy to maintain an old API even while you add features to the new API, and the runtime penalty is minimal because clients would be expected to call just one version of the API for any given event. (You can also see who’s calling the old API and ask them to change) But this is not true for events. If you change the body such that you…

Supporting multiple versions of an event schema is a solved problem. Apache Avro with a published schema hash in a message header is one solution.

https://avro.apache.org/

Post reply on HN