Live data from Hacker News

Building Microservices the Lean Way, part 2

blog.hubblehq.com

1–10 of 44 posts

Re: Building Microservices the Lean Way, part 2

#4
Hi Tom, I too have a django monolith. But, I hesitate to go down the microservices route, since I reuse alot of classes in what would become different services. Can you comment on how your class structure has changed, and how you have maximized (or not) code reuse?

Re: Building Microservices the Lean Way, part 2

#5
FWIW, I've found that building a robust and deep "API Gateway" is the key to making SOA/Microservices work. Otherwise, you end up with duplication and latency.

Routing and authentication are obvious candidates. It's also a good place to track stats and tag each request with a unique ID so you can trace it as it flows through your services.

By "deep", I mean that it should be application-aware. Caching is a good example. For many applications, url + querystring results in too many permutations. If the cache is aware, it can often use more meaningful keys. Additionally, events in the underlying services can be used to cache purge, which can result in a wicked cache hit ratio.

A more complex example has to do with duplication. Say you're building an ecomm platform. You have a service for search, and one for recommendations and one for the main catalog. They all need to return the same representation of a "product". Do you duplicate the logic? Do you tie them together and pay the latency and reliability price? No. Have them all just return IDS, and let the API Gateway hydrate the actual results. It's a form of API-aware server-side include and it works well.

Re: Building Microservices the Lean Way, part 2

#6

Hi Tom, I too have a django monolith. But, I hesitate to go down the microservices route, since I reuse alot of classes in what would become different services. Can you comment on how your class structure has changed, and how you have maximized (or not) code reuse?

What sort of classes do you mean? Views? Models? Other? For us it hasn't changed much. Our apps were pretty self-contained so that splitting them into separate services isn't very arduous.

Stuff that is shared between apps is often related to 3rd party integrations, which could be moved into a separate (often asynchronous) worker/ service. In reality most of these design choices are done on a case by case basis, based on time/ cost/ maintenance.

Re: Building Microservices the Lean Way, part 2

#7
post #5

FWIW, I've found that building a robust and deep "API Gateway" is the key to making SOA/Microservices work. Otherwise, you end up with duplication and latency. Routing and authentication are obvious candidates. It's also a good place to track stats and tag each request with a unique ID so you can trace it as it flows through your services. By "deep", I mean that it should be application-aware. Caching is a good examp…

This is really interesting. Would you consider writing a more in-depth post on this? I'd love to read it.

Re: Building Microservices the Lean Way, part 2

#8

Hi Tom, I too have a django monolith. But, I hesitate to go down the microservices route, since I reuse alot of classes in what would become different services. Can you comment on how your class structure has changed, and how you have maximized (or not) code reuse?

What sort of classes do you mean? Views? Models? Other? For us it hasn't changed much. Our apps were pretty self-contained so that splitting them into separate services isn't very arduous. Stuff that is shared between apps is often related to 3rd party integrations, which could be moved into a separate (often asynchronous) worker/ service. In reality most of these design choices are done on a case by case basis, base…

Yeah my main concern is for models. I can see it helps if you have distinct django apps already, in my case I have one main monolithic app. As an example I use elasticsearch, but I post process the results using models. ES is a service already do I really want to iolate some logic and build another service on top of that?

Re: Building Microservices the Lean Way, part 2

#10
post #5

FWIW, I've found that building a robust and deep "API Gateway" is the key to making SOA/Microservices work. Otherwise, you end up with duplication and latency. Routing and authentication are obvious candidates. It's also a good place to track stats and tag each request with a unique ID so you can trace it as it flows through your services. By "deep", I mean that it should be application-aware. Caching is a good examp…

This is really interesting. Would you consider writing a more in-depth post on this? I'd love to read it.

If you mean the SSI: http://openmymind.net/Practical-SOA-Hydration-Part-1/ http://openmymind.net/Practical-SOA-Hydration-Part-2/

I'm playing with it again on a new project, with a twist. Each item can be heavily personalized. Sticking with the ecomm example, the virgin product might look like

    {
      "id": "434p",
      "name": {"en": "..."},
    }
But we want to expand that, per user, with stuff like:

    "liked": true, 
    "bought": "22-1-2015"
We don't want to burden the clients with having to make multiple calls. I'm still working it out, but the services will continue to just return a list of product ids, and the API Gateway will now hydrate both the product and the personalized pieces. Something like:

   res = upstream(req)
   ids = extractIds(res.Body)
   products = getProducts(ids)
   personalized = getPersonalized(ids, currentUser)
   reply(merge(products, personalized))

Not critical, but worth pointing out that, for me, the API Gateway acts like a gigantic product cache with _every_ product in-memory. When a product changes, it gets an event and updates its cache. It isn't really a "cache", since there's never a miss. Trying to figure out if I can do the same with personalized data. (Even if you have tens millions of product, you can easily store it in memory).
Post reply on HN