Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone fully embraced an event-driven architecture?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 173 posts

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#141

Post by chhhris »

jimmaswell wrote: post #132
PinguTS wrote: post #61

Earlier quoted context omitted.

There is an example where highly distributed, event-driven systems are used. Everyday we are using them. It the cars. In a car there are many distributed ECUs(1) that communicate in an event driven system. It was tried to use cyclic communication. But all those attempts failed in the long run. Because cyclic communication would required that all the independent ECUs are synced to each other, which is a very hard prob…

Electronics can be more trouble than they're worth in cars. If they made modern cars with no electronics in the engine or controls, maybe I'd buy new cars. It's frustrating when my old Cherokee won't start because the antitheft system is having some kind of conniption for example, and I have to go through this ritual of locking and unlocking the doors, reconnecting the battery with the key in the ignition, flipping t…

I exclusively drive modern cars, and have never experienced anything along the lines of what you describe. So maybe the problem isn't with modern cars, but old Cherokees ;)

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#142

Post by jimmaswell »

jfoutz wrote: post #41

You're not going to like the answer, but I think it captures some of what you're getting at. Windows 95. Old style gui programming meant sitting in a loop, waiting for the next event, then handling it. You type a letter, there's a case switch, and the next character is rendered on the screen. Being able to copy a file and type at the same time was a big deal. You'd experience the dead letter queue when you moved a wi…

Aren't most "events" just an abstraction that's actually implemented by polling and/or an event queue at some level anyway, be it a library or the kernel?

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#143

Post by linseed_213 »

kieranmaine wrote: post #133
sumtechguy wrote: post #127

Earlier quoted context omitted.

kafka has a 'schema registry' service. Typically they use avro/json to define the schema of each message. Think they added a couple of new types in recent versions. When you define your consumer/producer you also tell avro what the schema registry service URL is. Basically it helps you keep your messages in spec. It also has the ability for versioning so you can have different versions of messages in flight on the sa…

Thanks for the reply. We're using a schema registry but we generally have our schemas spread across multiple repos depending on who owns the topic. I was wondering if they centralized all their schemas under a single repo. I'd like to do this, but I wanted to get some other opinions on the subject.

Evan can correct me if I'm wrong, but I believe it was one centralized service with all the schemas. You could view older versions and make new versions via the UI.

It was pretty user-friendly and managing schemas was straight forward. Haven't done anything similar since so no comparison point, but I thought it was fantastic and improved data quality a ton.

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#144

Post by MichaelMoser123 »

MichaelMoser123 wrote: post #62

nginx is fully event driven https://github.com/nginx/nginx here is a list of what nginx is using for event handling on the different operating systems: https://nginx.org/en/docs/events.html i don't know of any big project using the newer io_uring, does anyone know some big examples of io_uring usage?

https://github.com/CarterLi/nginx-io_uring here is a project that tries to use io_uring with nginx; you learn something every day...

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#145

Post by phaedrus »

I maintain a legacy application which was written in a fully event-driven way. As another commenter mentioned, native Windows programs work this way, but it is not just because this is a Win32 program. The original author(s) of this application also embraced a multi-threaded paradigm and use their own homegrown asynchronous serial message and event system (built on top of Windows messages).

It's terrible. The reason it's terrible is that you can't use function call graphs, single stepping, or call stacks to debug this application. Everything happens indirectly by one part of the application throwing a message in a bottle into the ocean, and another part of the application (running on another thread) finding the bottle at a later time. And every message is written in a different binary format (different memcpy'd structs) which is mutually intelligible to each sender-receiver pair and no one else.

Troubleshooting and understanding this system is more akin to endocrinology or ecology than math or engineering.

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#146

Post by nitwit005 »

Used it for fully transient services. If search died, it could be rebuilt from scratch. If chat died, data about active chats would be lost, but no one really cared.

The main issue with using it with other services was answering questions like "how do we restore a database backup?" or "what do we do if Kafka blows up, and we lose messages?". It's possible to create a design that won't get into an inconsistent state when bad things happen, but people tend to greatly underestimate how hard it is.

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#147

Post by nycjay »

sideway wrote: post #79
tjungblut wrote: post #71

Earlier quoted context omitted.

if you're using a queue like SQS and expect it to be ordered and exactly-once, you're in for a lot of surprise. If you need ordering, use a stream/log like Kinesis or Kafka.

Exactly-once is not a requirement but ordering is. I had Kafka and Kinesis in mind when writing this question but just in case you haven't seen it, there is a way to get ordering guarantees using SNS and SQS: https://aws.amazon.com/about-aws/whats-new/2020/10/amazon-sn...

Does that really guarantee in-order processing, or just that messages can be picked up in order. If you have multiple consumers on your queue and consumer A picks up message x from the head of the queue then consumer B picks up message y next, it is possible for y to get processed before x. Maybe consumer A is slow (gc pause?) for some reason, and now we are processing out of order, even though the queuing infrastructure does not see it. If you truly need to guarantee strict processing order (x must complete before you start processing y) I think you may need to build that into your app. Or, I misread, which is very possible.

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#148

Post by aynyc »

claytonjy wrote: post #125
aynyc wrote: post #99

Earlier quoted context omitted.

Event sourcing and event driven are two different things.

What is their relationship? I've been thinking of event-sourcing as a special case of event-driven: all event-sourced systems are event-driven, but the reverse is not true. Is this the generally accepted view or do folks view these as more orthogonal?

In my opinion, event sourcing is a data persistence strategy. I personally never heard that it's considered part of event-driven architecture, but given speed of change in that space, I wouldn't be surprised.

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#149

Post by bob1029 »

gwbas1c wrote: post #118

I've been working on a contract for 6 months where the architecture is microservices and queues. IMO: It's over-complicated. They can ship a change to a microservice while insulating the other services from risk, but that's just kicking the can for technical debt. What happens is that, if a service hasn't shipped in a few release cycles, when an update is made to that service, we often find latent bugs. Typically the…

We made the microservices mistake circa 2015-2016. Took us 3 years to recover and we lost some customers. Extremely valuable lessons were learned by all involved.

Re: Ask HN: Has anyone fully embraced an event-driven architecture?

#150

Post by monksy »

sagichmal wrote: post #24
monksy wrote: post #14

I have at a few companies now. It's great for processing data that goes beyond a single database call, data formatting and presenting something on a page. If you're chaining multiple microservices together you've made a very sloppy/poor mans version of this. (People tend not to account for downed services, maintence updates, client durability, etc) When you bring in technologies like Kafka to orchestrate this, you'll…

> When you bring in technologies like Kafka to orchestrate this, you'll end up with a more reliable system that you can fix if something goes wrong. It depends... If your services are servicing typical user requests, and expect responses in O( want backpressure, RPC semantics, etc. https://programmingisterrible.com/post/162346490883/how-do-y... If you're doing ETC/event semantics, the game is different. But very few…

> If your services are servicing typical user requests, and expect responses in O(Correct. Most cases you need to see what the current stored state is. Processing can happen in the background and update that state without being requested.

I'm assuming that you're architecture your system as having an ingest service that brings everything into the topic, multiple apps that'll refine/enhance, the data, and then an app that sends it out to a more permanent storage. Your service endpoint should look at the premant storage to make a response. (Or read and post out updates via a websocket if you want frequent updates to an existing page)

Kafka is not a network technology.

Post reply on HN