[deleted]
Ask HN: Has anyone fully embraced an event-driven architecture?
81–90 of 173 posts
Re: Ask HN: Has anyone fully embraced an event-driven architecture?
#82I've worked on a project that was fully event-driven realized as Microservices. (I think a few of the externally connected Microservices were event-driven as well but not all.) That was all roll-your-own without framework. So things could break. But the philosophy was more like: everything is written in a very lightweight and simple way. So if it breaks, it can be fixed swiftly. I've also seen a similar approach at another place. FWIW both places had unusually high availability requirements. ("Fail fast"...)
Re: Ask HN: Has anyone fully embraced an event-driven architecture?
#83Much of our codebase is python microservices communicating via Kafka. Once you get past the hurdle of getting kafka connected it's pretty reliable. We have a shared library for producing and consuming so we don't need to reinvent the wheel for new services. We also dump the resulting messages as rows into a database. It works very well.
What do you use to encode the messages? Avro, Proto or maybe just JSON? Most of the books and articles encourage the use of Avro, but I feel like the support in python isn't nearly as good as in other languages. Almost every solution operates on dictionaries instead of classes with annotations which doesn't seem like something you would want to use for a Py project in 2021...
Re: Ask HN: Has anyone fully embraced an event-driven architecture?
#84Not sure if there are any communities. My general advice is to invest as much as possible in a good logging solution, traceability, and just general things to make debugging easier. Come up with a way to replay events easily. You'll thank yourself everyday a bug or issue pops up.
I'm not sure you've understood EDA if you're suggesting that good logging is an alternative.
Re: Ask HN: Has anyone fully embraced an event-driven architecture?
#85There's no such architecture, much like there's no "MVC architecture" or "CQRS architecture". These are patterns that should be used specifically in time and space where and when pros outweigh cons.
Anyone calling themselves an architect, or an engineer, or even just a "good developer" would acknowledge that interaction patterns and concepts are contextual, not general idioms at the project or system level.
Speaking of them as "architectures" or embracing them, as in, doing everything the one holy way is only a crutch for people who are confused by what it means to define your system's architecture. And a silver bullet for consultants to sell you books and training.
There is a lot of empty hype and misconceptions around EDA, for example "it helps decouple services" is thrown around, which is nonsense to anyone who can analyze a system and knows what a dependency is (moving from "I tell you" from "you tell me" is not more decoupled, you just moved the coupling; likewise moving from "A tells B" to "everyone tells B and B tells everyone" as in event hubs is much more coupling, it plays the role of a global system variable basically).
Regarding dead letters, a most trivial answer is log and notify the stakeholders for unconsumed messages. That's the most general approach. Think about dead letter messages the same as exceptions that bubbled to the top of the stack. And when you can handle them more specifically, you do.
Re: Ask HN: Has anyone fully embraced an event-driven architecture?
#86Yes, I made my own open-source event driven platform: http://github.com/tinspin (rupy is the foundation and fuse is an example implementation tested with 350.000 users and 5 years uptime)
The learnings where 2-fold:
1) You need async-to-async capable db clients so that you use 4 threads (potentially on separate cores) for each browser server database roundtrip.
Since most databases don't have async capable clients I wrote my own database too: http://root.rupy.se
2) You should use a VM + GC language so that you can use atomic shared memory between cores, that way any core can handle any request efficiently (and access other users memory).
This part is very hard to prove in theory, but in practice I'm baffled by how well Java performs, you can find three quotes that I managed find here: https://github.com/tinspin/rupy/wiki
Finally getting threads to cooporate on things is hard and you cannot debug it with any tools, instead you have to use "trial and error" until is sort of works all the time.
Re: Ask HN: Has anyone fully embraced an event-driven architecture?
#87Earlier quoted context omitted.
As someone that worked at the company that taught Pete Hunt, and other early contributors to react this trick, I wouldn't call it a benefit, but the only workable solution to acceptable single page web application performance in an otherwise terrible development environment (browser javascript). Proper concurrency support and real mutexes would be way better than having a single execution thread for all your computat…
I don’t think I agree. I can’t think of many tasks I’ve ever tried to solve in programming where moving the work to another thread was the right answer. Usually things are too entangled, or too fast to bother. And there’s so much overhead in threading - from thread creation and destruction, message passing with mutexes or whatever and you need to figure out how to share data in a way that doesn’t accidentally give yo…
I've also found few use cases for workers in the browser, but as the parent pointed out that's partly because of the limited and sluggish communication between threads in javascript. Just to offer an opposite perspective, serverside, almost any modestly heavy processing I do with nodejs happens in worker threads. These are pooled so they don't need to spin up. Blocking node's event loop is never an option. Fine to access a big dataset with an async call, but if you need to crunch that data before passing it to the client you pretty much have to use workers. [edit] obviously assuming you're not calling some command line process, and you're trying to chew though some large rendering of data using JS.
Re: Ask HN: Has anyone fully embraced an event-driven architecture?
#88No. In my programs I try to hold John Carmack's advice, that there is no better understandable structure than a large program that you can read from start to end. He is talking about a main game loop, but I found that this advice holds. Nothing beats being able to step the function step by step and see all the variables.
Re: Ask HN: Has anyone fully embraced an event-driven architecture?
#89Earlier quoted context omitted.
I guess it's still harder to track down event emitters, but have you tried using bitbucket or GitHub code search to search all of your repos at once?
Yea, I have use GitHub search in a pinch and sometimes it is helpful enough to show me exactly where to look. Unfortunately, though, there are several events we emit that are many layers of string concatenation, so GitHub search may narrow it down to 4 or so places and I have to manually go from there.
Solution for that is using equivalent of user-agent - write it to message headers. Not sure if SQS supports it like Kafka.
I'm really surprised that not a lot of companies use this.
Re: Ask HN: Has anyone fully embraced an event-driven architecture?
#90The closest thing I know of is the Erlang/Elixir approach to program development. The BEAM VM that they're built on, is basically an instantiation of the actor programming model - a series of logical processes (services) that only communicate with each other through messages. Any state is held in an actor and you work with that state in an event driven way based on the messages you receive. I'll give a little peek at this below, but really you'd need to work with it to see how well it works at application scale.
In well architected Erlang/Elixir, most of your business logic will be written as pure functional code (which is gloriously easy to test), but then it is glued together at the boundary by GenServers (usually). GenServers are an abstraction over the BEAM primitives that makes the 'receive a message, update my state' thing very easy. The simplest handler might look like this:
def handle_call(:increment, _from, state) do
{:reply, state, state + 1}
end