11–20 of 44 posts
Hi Tom, I too have a django monolith. But, I hesitate to go down the microservices route, since I reuse alot of classes in what would become different services. Can you comment on how your class structure has changed, and how you have maximized (or not) code reuse?
I saw a SOA proponent build out a service for "products", instead of accessing them directly from the database in the application itself, and the result is a service that is fast on surface, but when you make 6 calls to it, it still adds up and is by far the biggest performance bottleneck.
I guess approaching your SOA as DB over HTTP and additional layers is to blame here :)
FWIW, I've found that building a robust and deep "API Gateway" is the key to making SOA/Microservices work. Otherwise, you end up with duplication and latency. Routing and authentication are obvious candidates. It's also a good place to track stats and tag each request with a unique ID so you can trace it as it flows through your services. By "deep", I mean that it should be application-aware. Caching is a good examp…
This is really interesting. Would you consider writing a more in-depth post on this? I'd love to read it.
https://github.com/facebook/Haxl https://www.youtube.com/watch?v=VVpmMfT8aYw http://getclump.io/
For example, let's I am working on service A, which depends on services B and C. Do I need to run all 3 apps and their data stores locally?
We currently will typically point A to the staging B and C. However, we have some long running jobs that A will initiate on B and B needs to post back to A when it's finished. This doesn't work when pointing to staging B.
For example, let's I am working on service A, which depends on services B and C. Do I need to run all 3 apps and their data stores locally?
We currently will typically point A to the staging B and C. However, we have some long running jobs that A will initiate on B and B needs to post back to A when it's finished. This doesn't work when pointing to staging B.
FWIW, I've found that building a robust and deep "API Gateway" is the key to making SOA/Microservices work. Otherwise, you end up with duplication and latency. Routing and authentication are obvious candidates. It's also a good place to track stats and tag each request with a unique ID so you can trace it as it flows through your services. By "deep", I mean that it should be application-aware. Caching is a good examp…
I consider services to fall into two groups:
1. Core/Technical services that are the master record for data and are responsible for data integrity, notifying others of changes, audit, etc... but ultimately store the data and the schemas of a piece of data, handle caching.
2. Composition/Orchestration/Business services that sit in front of the core services, and these contain business logic and when they operate on lists they really operate on IDs and do that ESI merge thing composing their collection from multiple calls from the core service.
The more I work on services the more I find this separation and layering of services extremely useful. It helps massively simplify the services that interact with data stores, and allows for quick development of new composition services to handle new needs of the business.
FWIW, I've found that building a robust and deep "API Gateway" is the key to making SOA/Microservices work. Otherwise, you end up with duplication and latency. Routing and authentication are obvious candidates. It's also a good place to track stats and tag each request with a unique ID so you can trace it as it flows through your services. By "deep", I mean that it should be application-aware. Caching is a good examp…
The risk here is that the API gateway gets too big, and its gravity starts attracting more and more functionality, until you have a brand new monolithic application.
I like the gateway approach, but generally I want to have multiple gateways, each serving as a "Microservice Facade" for a particular application or small set of applications. New applications get new Facades.
There ends up being some duplication across the facades, but basically I think "so be it". As long as the core services are factored out and own their business logic, I don't mind if the consuming applications have some duplication.
So here's a question we've been talking about at my office. When developing a micro-service on your development machine, do you need to run the whole stack or just the service you're working on? For example, let's I am working on service A, which depends on services B and C. Do I need to run all 3 apps and their data stores locally? We currently will typically point A to the staging B and C. However, we have some lon…
A stopgap solution is to use something like an Actor[2] model, which schedules actors on an ActorSystem and to clone the context/scheduler/system for each client ID. As long as your actors are fairly sane, this should be fairly lightweight. Then just shut down the actors for a given client (actorSystem.shutdown() under Akka), either after a time or by having a client send a Shutdown message (or both).
[1]: http://wym.io/ [2]: http://akka.io/
FWIW, I've found that building a robust and deep "API Gateway" is the key to making SOA/Microservices work. Otherwise, you end up with duplication and latency. Routing and authentication are obvious candidates. It's also a good place to track stats and tag each request with a unique ID so you can trace it as it flows through your services. By "deep", I mean that it should be application-aware. Caching is a good examp…
Three useful definitions[1] for microservices are:
1) collaborating
2) independently deployable (Martin Fowler [2])
3) globally-aware
Having an API Gateway makes them follow none of these meaningfully, as:
1) the services are collaborated _upon_ rather than doing so with each other (see Clump[3], not itself a bad idea)
2) updating a microservice will many times require a redeploy of the API gateway and for everyone to update their connections to it
3) any functionality you put in the gateway becomes encapsulated in a way tailored to application-global needs, which is pretty inimical to modular components, so you'd better not talk through the gateway except where you absolutely must!
Having orchestration/aggregation services as buro9 mentions in a sibling post is one good solution, satisfying all of those three points much more cleanly.
The danger arises when you start to think of your gateway(s) as layers in front of your system rather than just more microservices in it.
[1]: (context) https://vimeo.com/118895501#t=48s
[2]: http://martinfowler.com/articles/microservices.html
[3]: http://getclump.io/
So here's a question we've been talking about at my office. When developing a micro-service on your development machine, do you need to run the whole stack or just the service you're working on? For example, let's I am working on service A, which depends on services B and C. Do I need to run all 3 apps and their data stores locally? We currently will typically point A to the staging B and C. However, we have some lon…
If I just wanted to poke around a running system to see how things interact manually, yeah I'd run everything locally. Probably using Docker images for each dependency service and Vagrant to manage the suite of those images, to preserve my sanity.