Logging sucks

I wonder if one might solve this by using an accumulator that merges objects as they are emitted based on some ID (i.e. request ID say) and then ether emits the object on normal execution or a global exception handler emits it on error...?

YEAH! If its less than a 100TB as second I don't even get out of bed!

Your objection, as I understand it, is some combination of "no true Scotsman" combined with complaints about wide events themselves.

To the first point (no true Scotsman), I really don't think that applies in the slightest. The post I'm replying to said (paraphrasing) that middle-layer observability is hard with wide events and easy with logs. My counter is that the objection has nothing to do with wide events vs logs, since in both scenarios you can choose to include or omit more information with the same syntactic (and similar runtime overhead) ease. I think that's qualitatively different from other NTS arguments like TDD, in that their complaint is "I don't have enough information if I don't send it somewhere" and my objection is just "have you tried sending it somewhere?" There isn't an infinite stream of counter-arguments about holding the tool wrong; there's the very dumbest suggestion a rubber duck might possibly provide about their particular complaint, which is fully and easily compatible with wide events in every incarnation I've seen.

To the second point (wide events aren't especially useful and/or suck), I think a proper argument for them is a bit more nuanced (and I agree that they aren't a panacea and aren't without their drawbacks). I'll devote the rest of my (hopefully brief) comment to this idea.

1. Your counter-example falls prey to the same flaw as the post I responded to. If you want information then just send that information somewhere. Wide events don't stop you from gathering data you care about. If you need a requestID then it likely exists in the event already. If you need a message then _hopefully_ that's reasonably encoded in your choice of sub-field, and if it's not then you're free to tack on that sort of metadata as well.

2. Your next objection is about the wide event being so context-rich as to be a problem in its own right. I'm sympathetic to that issue, but normal logging doesn't avoid it. It takes exactly one production issue where you can't tie together related events (or else can tie them together but only via hacks which sometimes merge unrelated events with similar strings) for you to realize that completely disjoint log lines aren't exactly a safe fallback. If you have so much context-dependent complexity that a wide event is hard to interpret then linear logs are going to be a pain in the ass as well.

Mildly addressing the _actual_ pros and cons: Logs and wide events are both capapable of transmitting the same information. One reasonable frame of reference is viewing wide events as "pre-joined" with a side helping of compiler enforcement of the structure.

It's trivially possible to produce two log lines in unrelated parts of a code base which no possible parser can disambiguate. That's not (usually) possible when you have some data specification (your wide event) mediating the madness.

It's sometimes possible with normal logs to join on things which matter (as in your requestID example), but it's always possible with wide events since the relevant joins are executed by construction (also substantially more cheaply than a post-hoc join). Moreover, when you have to sub-sample, wide events give an easy strategy for ensuring your joins work (you sub-sample at a wide event level rather than a log-line level) -- it's not required; I've worked on systems with a "log seed" or whatever which manage that joinability in the face of sub-sampling, but it's more likely to "just work" with wide events.

The real argument in favor of wide events, IMO, is that it encourages returning information a caller is likely to care about at every level of the stack. You don't just get potentially slightly better logs; you're able to leverage the information in better tests and other hooks into the system in question. Parsing logs for every little one-off task sucks, and systems designed to be treated that way tend to suck and be nearly impossible to modify as desired if you actually have to interact with "logs" programatically.

It's still just one design choice in a sea of other tradeoffs (and one I'm only half-heartledly pursuing at $WORK since we definitely have some constraints which are solved by neither wide events nor traditional logging), BUT:

1. My argument against some random person's choice of counter-argument is perfectly sound. Nothing they said depended on wide events in the slightest, which was my core complaint, and I'm very mildly offended that anyone capable of writing something as otherwise sane and structured as your response would think otherwise.

2. Wide events do have a purpose, and your response doesn't seem to recognize that point in the design space. TFA wasn't the most enjoyable thing I've ever read, but I don't think the core ideas were that opaque, and I don't think a moment's consideration of carry-on implications would be out of the question either. I could be very wrong about the requisite background to understand the article or something, but I'm surprised to see responses of any nature which engage with irrelevant minutea rather than the subset of core benefits TFA chose to highlight (and I'm even more surprised to see anything in favor of or against wide events given my stated position that I care more about the faulty argument against them than whether they're good or bad)..

Most of that one still rings very true to me. I particularly liked this section:

> Let’s start here: hiring engineers is not a process of “picking the best person for the job”. Hiring engineers is about composing teams. The smallest unit of software ownership is not the individual, it’s the team. Only teams can own, build, and maintain a corpus of software. It is inherently a collaborative, cooperative activity.

Diagnosing "failure" in other people is indeed rude, even if you privately consider it true and an appropriate characterization. It's worse if you do that after jumping to the conclusion that somebody else has not considered something, because they have a different opinion than you. At least that's my conclusion of why you wrote that. (And this paragraph is my return offering of genuine and valuable wisdom.)

Of course you can keep everything together, in just very few large parts, or even a monolith. I've not said otherwise.

My point is that "architecture" is orthogonal to the question of "monolith vs separate services"; the difference there is not architecture, but in cohesion and flexibility.

If you do things right, even inside a monolith you will have things clearly separated into different concerns, with clean interfaces. There are natural service boundaries in your code. (If there aren't, in a system like this, you and the business are in for a world of pain.)

The idea is that you can put network IO between these service boundaries, to trade off cohesion and speed at these boundaries for flexibility between them, which can make the system easier to work with.

Different parts of your system will have different requirements, in terms of criticality, performance and availability; some need more compute, others do more IO, are busy at different times, talk to different special or less special databases. This means they may have different sweet spots for various trade-offs when developing and running them.

For example, you can (can!) use different languages to implement critical components or less critical ones, which gives you a bigger pool to hire competent developers from; competent as developers, but also in the respective business domain. This can help your company off the ground.

(Your IoT and bike people are comfortable in Rust. Payments is doing Python, because they're used to waiting, and also they are the people you found who actually know not to use floats for money and all the other secrets.)

You can scale up one part of your system that needs fast compute without also paying for the part that needs a lot of memory, or some parts of your service can run on cheap spot instaces, while others benefit from a more stable environment.

You can deploy your BI service without taking down everything when the new initialization code starts crash-looping.

(You recover quickly, but in the meantime a lot of your IoT boxes got lonely are now trying to reconnect, which triggers a stampede on your monolith, you need to scale up quickly to keep the important functions running, but the invoicing code fetches a WDSL file from a slow government SOAP service, which is now down, and your cache entry's TTL expired, and you don't even need more invoicing right now... The point is, you have a big system, things happen, and fault lines between components are useful.)

It's a trade-off, in the end.

Do you need 15 services? You already have them. They're not even "micro", just each minding their own part of the business domain. But do they all need their own self-contained server? Probably not, but you might be better off with more than just one single monolith.

But I would not automatically bat an eye to find that somebody separated these whatever-teen services. I don't see that as a grievous error per se, but potentially as the result of valid decisions and trade-offs. The real job is to properly separate these concerns, whether they then live in a monolith or not.

And that's why that request may well touch so many services.