Breaking down an application into micro services allows me to keep a clearer picture of the entire application and how it should work. Not to mention the advantage of evolving the services individually without affecting the entire system. I've started a little library in Python for writing small services. It leverages nanomsg. link: https://github.com/walkr/nanoservice
Microservices - 72 resources
41–48 of 48 posts
Re: Microservices - 72 resources
#42Earlier quoted context omitted.
That's often not practical. Say you're a shop front. If you have a goods purchase service and a goods delivery service, you want to reserve the good (limited quantity) and reserve a time slot for transport (limited number of deliveries a day) before you charge the customer, and make the "commit". And if you'd merge two separate companies into one uber-service, you've just created more problems than you've solved.
Atomic transactions are one of several strategies for handling this problem, and not always the right answer. Starbucks is a good analogy ( http://www.eaipatterns.com/ramblings/18_starbucks.html ). If Starbucks were transactional, you would have to stand there at with the money on the counter and wait until your drink was finished, then exchange at the exact same time. Even the mental image is ridiculous. In practice…
I don't know what practice you're referring to, but unlike commodity coffee drinks paid for in cash 1) CC refunds are not free, and they're not cheap in volume at all if you intend to do it casually during normal operation 2) not all goods are standardized and available in large quantities.
Your examples are all over the place. Ticketmaster is an example of a reservation system similar to one I was trying to give an example of (two step commit). A resource is locked, the lock is held for a short period while collecting answers from the other subsystems (in this case, payment gateway), and then a final commit is issued (or a rollback is issued).
Airline overselling isn't done because it was some microservice design dogma about how great inconsistent state is, but because every seat costs the airline a fortune if left empty, and a certain % of passengers cancel or reschedule their tickets, and the airline is trying to arrive at an airplane with as few empty seats as possible. Having your tickets canceled is certainly not something that happens "often", thank god, but it does happen as a result of that tradeoff.
But if I reserve and buy my seats online for a cinema movie, and then I go with company and get handed my money back because "it's practical", I'll make a scene. And so no one implements cinema ticket reservation this way.
For bank overdrafting, it's a very special case - your money is a number in a computer, and the bank owns that computer. They make the rules... so they did. It's easy to mess around with numbers like that. Bank account overdrafting is probably the biggest exception of them all as no physical products and services are involved. No one's going to have their lawn un-mowed because the bank allowed your account to overdraft.
The only common thing between your examples is that they're driven by business concerns, not some ivory tower concern about service design. And this is why they're so different, and reserving resources is and will remain a common practice for many, as long as the business logic calls for it. There's nothing wrong about it.
Re: Microservices - 72 resources
#43Why do we need a new term for this? Isn't this just Unix philosophy applied to SaaS? No need to learn about "micro services" when you can learn about Unix/Linux and why it's built the way it is, then apply that knowledge to your SaaS product.
First, a key part of the Unix philosophy is the pipes. Programs do not really communicate without them. The closes thing I have seen in a microservice architecture is some magical bus, which is not easy to implement well, thanks to the fun of distributed systems: We don't spent time wondering if grep is going to get the data twice, or not at all, do we?
A second difference is that, in the unix world, we fully accept the concept of glue. Along with the reusable tools of the world, we have all the code that we know is not going to be reusable, that lets us do the task at hand. Depending on how hard it is, we might use something from bash to Perl. In a microservice architecture, such glue, if created, would be the actual logic of your application! So the moment you start adding glue, then the attention will quickly go to the glue, like the C gets pretty big in MVC.
Now, personally I am no believer in microservices as the panacea. The big advantage of the unix world is the reusable components. In a typical business application, the one we tend to get paid to do, how much code is actually stable enough, and reusable enough, that we really believe that its interface will last us a long time? In my experience, it's very little.
Where I have seen microservice-like architectures pull their weight is in actor systems, as a way to handle massive parallelism. But, just like many NoSQL implementations, they are solving a problem that many people would love to have, and very few people actually do. So we see startups building architectures in semi-exotic languages to handle hundreds of thousands of connections a second, when their actual volume of work could be handled with a boring Java app pointing to a Postgres database.
Re: Microservices - 72 resources
#44What do people recommend for learning about SOA / microservices? I liked Patterns of Enterprise Application Architecture and Enterprise integration patterns, but didn't have a great experience with the SOA design with Rails book - it felt much to focused on very basic details (e.g., here is how you consume JSON).
I tried to find SOA-related literature but most of it was impenetrable, enterprise Java type stuff. So we just kinda made it up as we went along, splitting off any services that seemed like it'd work, and that went really well.
Re: Microservices - 72 resources
#45Earlier quoted context omitted.
Atomic transactions are one of several strategies for handling this problem, and not always the right answer. Starbucks is a good analogy ( http://www.eaipatterns.com/ramblings/18_starbucks.html ). If Starbucks were transactional, you would have to stand there at with the money on the counter and wait until your drink was finished, then exchange at the exact same time. Even the mental image is ridiculous. In practice…
> "In practice, it makes much more sense to allow the system to enter certain inconsistent states and then remediate them. Out of an ingredient? Refund the money. Can't pay? Pull the drink out of the queue, or just write it off. This can involve some cost, but less cost than than the throughput you'd lose by enforcing consistency." I don't know what practice you're referring to, but unlike commodity coffee drinks pai…
Re: Microservices - 72 resources
#46Earlier quoted context omitted.
Atomic transactions are one of several strategies for handling this problem, and not always the right answer. Starbucks is a good analogy ( http://www.eaipatterns.com/ramblings/18_starbucks.html ). If Starbucks were transactional, you would have to stand there at with the money on the counter and wait until your drink was finished, then exchange at the exact same time. Even the mental image is ridiculous. In practice…
> "In practice, it makes much more sense to allow the system to enter certain inconsistent states and then remediate them. Out of an ingredient? Refund the money. Can't pay? Pull the drink out of the queue, or just write it off. This can involve some cost, but less cost than than the throughput you'd lose by enforcing consistency." I don't know what practice you're referring to, but unlike commodity coffee drinks pai…
In the Ticketmaster example, I would be able to get a Ticket resource assigned to me before paying for it. But the ticket would be "locked" (in the sense that I couldn't print the barcode) until the Payments service marked it as okay, which would indeed be an example of one service locking another's resource.
I thought you were claiming that the action needed to be performed in a single database transaction, but you don't need or want isolation here. You want the world to observe inconsistent state in this case (i.e. no one sees my seat as "available" while I'm fumbling around with my credit card number). In which case it's perfectly practical to implement transactionality at the application layer, between microservices, rather than at the database level. But that point was never in contention. I apologize.
Obvviously there's not a dogmatic preference for inconsistent state, just an acknowledgement that ACID properties are not necessarily needed as often as some people think they are.
Also, obviously, you have to be careful with your failure modes. Charging someone without delivering shouldn't happen often but you don't need to design your entire infrastructure around making it impossible, given that refunds (and simply not capturing the charge you authorized) are both relatively easy to do, even if you want to avoid doing them all the time.
Re: Microservices - 72 resources
#47Breaking down an application into micro services allows me to keep a clearer picture of the entire application and how it should work. Not to mention the advantage of evolving the services individually without affecting the entire system. I've started a little library in Python for writing small services. It leverages nanomsg. link: https://github.com/walkr/nanoservice
Why have you chosen not to use HTTP and all the ease of use and interoperability advantages it confers?
Re: Microservices - 72 resources
#48Earlier quoted context omitted.
> "In practice, it makes much more sense to allow the system to enter certain inconsistent states and then remediate them. Out of an ingredient? Refund the money. Can't pay? Pull the drink out of the queue, or just write it off. This can involve some cost, but less cost than than the throughput you'd lose by enforcing consistency." I don't know what practice you're referring to, but unlike commodity coffee drinks pai…
I apologize. For some reason I thought you were attacking the viability of microservice architecture because you can't do database-level transactions across several backends. You were arguing that services will sometimes need to lock each other, and I agree with that. In the Ticketmaster example, I would be able to get a Ticket resource assigned to me before paying for it. But the ticket would be "locked" (in the sen…
I really don't feel there's a need for us to separate "native" database transactions and app-level transactions. They're both implemented using the same underlying principles. But I've noticed people see a huge difference between them in blog posts, articles and conversations.
I think it reveals a kind of thinking that database transactions look like magic, while those we roll ourselves... we see all the ugly parts of the sausage factory there, and it no longer feels as "atomic" or magical as what databases expose as an encapsulated abstraction.