Live data from Hacker News

Ask HN: Has anyone fully embraced an event-driven architecture?

news.ycombinator.com

61–70 of 173 posts

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#61
post #41

You're not going to like the answer, but I think it captures some of what you're getting at. Windows 95. Old style gui programming meant sitting in a loop, waiting for the next event, then handling it. You type a letter, there's a case switch, and the next character is rendered on the screen. Being able to copy a file and type at the same time was a big deal. You'd experience the dead letter queue when you moved a wi…

I fully agree with this, also that's still quite common in the embedded world. The user presses a button which sets a hardware event flag. CPU wakes up from sleep, checks all event flags, handles them, clears the interrupt bits and goes back to sleep. But using events like this requires a very tight integration between event producer and consumer, so I don't think this will translate well to distributed systems or mi…

There is an example where highly distributed, event-driven systems are used. Everyday we are using them. It the cars.

In a car there are many distributed ECUs(1) that communicate in an event driven system. It was tried to use cyclic communication. But all those attempts failed in the long run. Because cyclic communication would required that all the independent ECUs are synced to each other, which is a very hard problem. That is the reason why everybody moved away from cyclic communication.

That said to help the development in automotive a middleware have been developed and used to help the development of such event driven systems. You develop your functions and define how the signals are routed between those functions. The someone later/independent decides on how the functions are distributed on the different ECUs depending the available resources. The middleware then takes care of the correct routing of the signals. Everything is event driven.

(1) Electronic Control Units

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#62
nginx is fully event driven https://github.com/nginx/nginx here is a list of what nginx is using for event handling on the different operating systems: https://nginx.org/en/docs/events.html

i don't know of any big project using the newer io_uring, does anyone know some big examples of io_uring usage?

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#63
post #41

You're not going to like the answer, but I think it captures some of what you're getting at. Windows 95. Old style gui programming meant sitting in a loop, waiting for the next event, then handling it. You type a letter, there's a case switch, and the next character is rendered on the screen. Being able to copy a file and type at the same time was a big deal. You'd experience the dead letter queue when you moved a wi…

> Have a plan for building pipelines a->b->c->d. also have a plan for fan out a->b & a->c & a->d

How does that look like? Do you mean to learn how this is done with the CI of choice, create helper functions or are there concrete steps that you would recommend? I'm new to this and would appreciate any feedback.

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#64
post #41

You're not going to like the answer, but I think it captures some of what you're getting at. Windows 95. Old style gui programming meant sitting in a loop, waiting for the next event, then handling it. You type a letter, there's a case switch, and the next character is rendered on the screen. Being able to copy a file and type at the same time was a big deal. You'd experience the dead letter queue when you moved a wi…

I fully agree with this, also that's still quite common in the embedded world. The user presses a button which sets a hardware event flag. CPU wakes up from sleep, checks all event flags, handles them, clears the interrupt bits and goes back to sleep. But using events like this requires a very tight integration between event producer and consumer, so I don't think this will translate well to distributed systems or mi…

[deleted]

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#65
> Every event-driven architectural pattern I've read about can quite easily fall apart and I have yet to find satisfying answers on what to do when things go south.

Just a kind request from someone living in the southern hemisphere not to use "south" as a synonym for "bad" or "fail".

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#66

Like others have said, it is just one tool in the tool box. We used Kafka for event-driven micro services quite a bit at Uber. I lead the team that owned schemas on Kafka there for a while. We just did not accept breaking schema changes within the topic. Same as you would expect from any other public-facing API. We also didnt allow multiplexing the topic with multiple schemas. This wasn’t just because it made my life…

Thanks for your detailed answer, really appreciate it.

Two follow up questions if you don't mind me asking, even though I understand you were not on the publishing side:

1. Do you know if changes in the org structure (e.g. when uber was growing fast and - I guess - new teams/product were created and existing teams/products were split) had significant effect on the schemas that had been published since then? For example, when a service is split into two and the dataset of the original service is now distributed, what pattern have you seen working sufficiently well for not breaking everyone downstream?

2. Did you have strong guidelines on how to structure events? Were they entity-based with each message carrying a snapshot of the state of the entities or action-based describing the business logic that occurred? Maybe both?

And yes, one of the books I'm talking about is indeed Designing Data Intensive Applications and I fully agree with you that it's a fantastic piece of work.

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#67
I recently got recommended this video, haven't watched it yet but given that the speaker is Kleppmann I suspect it will be very helpful

Thinking in Events: From Databases to Distributed Collaboration Software

https://www.youtube.com/watch?v=ePHpAPacOdI&list=WL&index=1&...

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#68
post #61

Earlier quoted context omitted.

I fully agree with this, also that's still quite common in the embedded world. The user presses a button which sets a hardware event flag. CPU wakes up from sleep, checks all event flags, handles them, clears the interrupt bits and goes back to sleep. But using events like this requires a very tight integration between event producer and consumer, so I don't think this will translate well to distributed systems or mi…

There is an example where highly distributed, event-driven systems are used. Everyday we are using them. It the cars. In a car there are many distributed ECUs(1) that communicate in an event driven system. It was tried to use cyclic communication. But all those attempts failed in the long run. Because cyclic communication would required that all the independent ECUs are synced to each other, which is a very hard prob…

[deleted]

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#69
post #9

Where I currently work we are all in on event-driven architecture. For our DLQs, we have alerts on when the queue is growing in size or if messages are in the queue too long. When those alerts come in, we manually move the messages back to the normal queue for reprocessing and if they get DLQed again after that we will look into the reason it is failing. One of the benefits of this architecture for us is the ability…

Thanks for your answer, it really helps.

Does moving the DLQ messages back to the normal queue mean that all consumers can deal with out-of-order scenarios?

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#70

Not for a company, but I've embraced it pretty hard for my home automation. It's sort of the hammer I hit everything hard enough with until it looks like a nail by making everything go through the MQTT broker. The website? A static json blob describes interesting MQTT topics, and opens a MQTT over websocket connection to read/write any state. Zigbee, et al.? Translate to MQTT. Reporting? Daemon that listens to all to…

MQTT at home is a real boon. I have mine chugging away in a raspi in a docker container and that drives everything like the plant watering. I also use it to collect metrics and do alerting with influx+grafana.
Post reply on HN