Live data from Hacker News

How we build microservices

blog.yourkarma.com

21–30 of 42 posts

Re: How we build microservices

#21

Is there a difference between microservices and SOA? I've used architectures just like this in the past but never heard of "microservices" until recently.

No difference, just a new name. Not so bad I guess, however I fear it will cropper when vendors leap on the bandwagon again and start sell inappropriate tools and frameworks badged as Microservice and we can start the process of renaming again.

Re: How we build microservices

#22
post #18

The shipment app listens to the messaging system, sees an order take place, looks at the details, and says, "Okay, I need to send two boxes to this person." Any other services interested in an order happening can do whatever they need to with the event in their own queues, and the store API doesn't need to worry about it. I'm curious how you guarantee that all the systems that need to see an event, will actually see…

Each app gets their own queue. So there is a queue for the shipment app and for the mailer app. When an event is published to SNS, it gets added to both queues.

Re: How we build microservices

#23
post #18

The shipment app listens to the messaging system, sees an order take place, looks at the details, and says, "Okay, I need to send two boxes to this person." Any other services interested in an order happening can do whatever they need to with the event in their own queues, and the store API doesn't need to worry about it. I'm curious how you guarantee that all the systems that need to see an event, will actually see…

Each app gets their own queue. So there is a queue for the shipment app and for the mailer app. When an event is published to SNS, it gets added to both queues.

[deleted]

Re: How we build microservices

#24
post #18

The shipment app listens to the messaging system, sees an order take place, looks at the details, and says, "Okay, I need to send two boxes to this person." Any other services interested in an order happening can do whatever they need to with the event in their own queues, and the store API doesn't need to worry about it. I'm curious how you guarantee that all the systems that need to see an event, will actually see…

Each app gets their own queue. So there is a queue for the shipment app and for the mailer app. When an event is published to SNS, it gets added to both queues.

I see. So you never add a listener, without also modifying the emitter to send to an additional queue.

Re: How we build microservices

#25
post #18

The shipment app listens to the messaging system, sees an order take place, looks at the details, and says, "Okay, I need to send two boxes to this person." Any other services interested in an order happening can do whatever they need to with the event in their own queues, and the store API doesn't need to worry about it. I'm curious how you guarantee that all the systems that need to see an event, will actually see…

I believe each service has it's own queue, so producing an event would "fan-out" and add the event to each queue. That was my interpretation of why they were using both SNS and SQS - but I could be mistaken. If a particular service is not interested in a given event, it can just pluck it out of the queue and move onto processing the next message.

Re: How we build microservices

#26

The text makes it seem like they're using SQS for many things, but the diagram only shows it between Orders and Shipments. I guess everything else communicates through the database and HTTP requests? Using SQS means you're either polling or long-polling with hangups every 20 seconds. This seems pretty shitty to me. Also how do you structure one of these services around polling or long-polling to get its info? It soun…

We use SQS for basically everything that happens asynchronously to the main flow. So when a user signs up there are a bunch of things that need to happen straight away (create an account, give 100MB to every new user), these go via HTTP. The others (email, give the owner of a Karma an additional 100MB and a push notification) goes through SNS and SQS.

All these async things don't impact the main flow of the user, so it's no problem that it isn't instantaneous. We are running background processes that do this.

We only use Rails for our frontend applications, they don't contain any business logic. Our backend processes are usually Sinatra for HTTP requests, or custom daemons.

Re: How we build microservices

#27

Is there a difference between microservices and SOA? I've used architectures just like this in the past but never heard of "microservices" until recently.

AFAICT, "microservices" is exactly SOA in its original sense; my guess is the rename is because SOA has become so attached to particular XML-based implementations and standards (e.g., the WS-* series of standards).

This. Also SOA means STD in Dutch, so we don't want to use it that often ;)

Re: How we build microservices

#28

Earlier quoted context omitted.

Yup we are. There might be multiple applications interested in the event and they each get their own queue.

Have you looked into a "real" messaging queue for this, like RabbitMQ or another AMQP based system for this? Or are you bound to SNS/SQS for a particular reason?

Why isn't SNS/SQS a "real" messaging queue ?

Re: How we build microservices

#29
post #18

The shipment app listens to the messaging system, sees an order take place, looks at the details, and says, "Okay, I need to send two boxes to this person." Any other services interested in an order happening can do whatever they need to with the event in their own queues, and the store API doesn't need to worry about it. I'm curious how you guarantee that all the systems that need to see an event, will actually see…

SQS is very simple and the solution here requires that the publisher knows all the listeners.

If you use RabbitMQ (which are we using in our own microservice architecture), you can use fanout exchanges based on topics to accomplish the same thing more elegantly.

In this topology, every message sent to that exchange gets copied to any queue bound to that exchange.

This system also supports routing via "topics", which are paths that support wildcards; the publisher can publish to "foo.bar.baz", and queues can bind to the exchange using the routing key "foo.*.baz", for example.

We use this to listen to specific events, eg. a specific app is associated with content under the path "someapp.someclient". A data store publishes modification events with "create", "update" or "delete" followed by the path; so the the app, to get the stream of updates, simply listens to "create.someapp.someclient.#", "update.someapp.someclient.#" and "delete.someapp.someclient.#".

Re: How we build microservices

#30
post #18

The shipment app listens to the messaging system, sees an order take place, looks at the details, and says, "Okay, I need to send two boxes to this person." Any other services interested in an order happening can do whatever they need to with the event in their own queues, and the store API doesn't need to worry about it. I'm curious how you guarantee that all the systems that need to see an event, will actually see…

SQS is very simple and the solution here requires that the publisher knows all the listeners. If you use RabbitMQ (which are we using in our own microservice architecture), you can use fanout exchanges based on topics to accomplish the same thing more elegantly. In this topology, every message sent to that exchange gets copied to any queue bound to that exchange. This system also supports routing via "topics", which…

You can use SNS to act as a fanout exchange. Then you just need to subscribe all your SQS queues to the SNS topic. Events get pushed to the SNS topic, and SNS will distribute them to all the queues for you with the pusher needing to know anything about the subscribers.
Post reply on HN