Live data from Hacker News

Ask HN: What's your biggest struggle with Microservices?

news.ycombinator.com

71–77 of 77 posts

Re: Ask HN: What's your biggest struggle with Microservices?

#71
post #13

We started with a monolith which made determining service boundaries that much easier. What we struggled with the most is the concept of sharing data between services. We started off with publish/subscribe method which quickly became very strenuous to maintain. Ended up switching to plain service-to-service communication over HTTP/S. This allowed us to focus on resiliency and performance of each micro-service more so…

"Ended up switching to plain service-to-service communication over HTTP/S." That statement assumes this picture: ServiceA -> Network (https) -> ServiceB I'm curious, what happens to the message being sent from ServiceA if Network or ServiceB is down?

We have a fraction of inter-service communication still flowing through pub/sub via RabbitMQ. This is typically done in cases where we absolutely cannot lose a message. Most of the communication, however, is via https with each microservice in the mix providing guaranteed high uptime, reasonable response time and some way of handling various failure conditions (ie fallback to cache, exponential backoff and etc). So while you do lose your message in this scenario, you have to have a very catastrophic network failure for this to have a wide-spread impact. We occasionally see blips which are dealt with via retries. So this has worked out fairly well at our not-so-crazy scale. I personally believe there is no right or wrong here. It all comes down to what your team is comfortable with. I think pub/sub is a great approach if you're willing to sink some time into it. Synchronous communication also has some very interesting development by companies like Netflix (Ribbon?).

Re: Ask HN: What's your biggest struggle with Microservices?

#72
post #62
post #26

Earlier quoted context omitted.

This is interesting. Can you provide an example - where it wouldn't have made sense to collapse two such interacting services into one? My understanding is that an important litmus test for carving up microservice boundaries is ensuring 'true' isolation/separation of concerns. Admittedly, I don't have enough practical experience to know if this is a misguided assumption.

Someone here mentioned a great rule of thumb that addresses most of issues here: one team = one service.

This is kind of weird rule since a MS is said to contain 1k loc. And having a full team (other than one guy) for 1k loc is... well how can you screw up so little code? The hard part isn't the teams. The hard part is to know what a MS should contain and still be useful.

Re: Ask HN: What's your biggest struggle with Microservices?

#74
One big challenge I've found is the balance between standardization vs autonomy. E.g. monitoring, log formats, deployment, service discovery, etc are things that can benefit from being standardized across all services, but implementing it can be tricky. Too much autonomy and you end up reinventing the wheel (inevitably with variations) - too much standardization (e.g. by providing centralized libraries) can make development slower and create dependency hell.

Re: Ask HN: What's your biggest struggle with Microservices?

#75
post #74

One big challenge I've found is the balance between standardization vs autonomy. E.g. monitoring, log formats, deployment, service discovery, etc are things that can benefit from being standardized across all services, but implementing it can be tricky. Too much autonomy and you end up reinventing the wheel (inevitably with variations) - too much standardization (e.g. by providing centralized libraries) can make deve…

very good point. maybe this is where service mesh (istio - envoy, linkerd) could be useful.

Re: Ask HN: What's your biggest struggle with Microservices?

#76

Earlier quoted context omitted.

That's a nice rule of thumb but it's radically different from the team I work on. We run half of one logical service (visible to users outside the team), but that logical service is implemented with a dozen or more microservices internally. We have user-facing services which need high priority, different flavors of batch jobs which need to be orchestrated and prioritized, and various other pieces of infrastructure. T…

Yeah I could totally see how if you have very strict uptime requirements and you want to allow different pieces of the infrastructure to be able go down at different times then it's an exception to the rule. Just for every team that I see that has a good use case for micro services, and does the hard work of instrumentation and deployment, I see 8 teams that go with microservices because they think it's a magic bulle…

In this case, we don't have strict uptime requirements. But there are enough times where our integration tests don't catch some kind of error, and it's nice that the service doesn't have to go completely down for that.

It's also a lot easier to prioritize process scheduling than it is to prioritize thread scheduling.

Re: Ask HN: What's your biggest struggle with Microservices?

#77

Earlier quoted context omitted.

Sure, but peeling off something that has radically different scaling properties is easier if you put a "web service" in the way.

I don't understand why putting a web service in front of mysql and a key value store makes scaling easier. Would you mind explaining?

Simple. Most of the database is "large" in terms of data (say 50M rows) but that database gets maybe 10,000 updates a day. The read load is well-controlled with caching.

One table is small in terms of data but involves an interactive service that might generate 50 updates/sec at peak times.

With the "hot" service implemented on top of a key-value store, the database is the ultimate commodity, I have many choices such as in-memory with logging, off-heap storage, distributed key-value stores, etc.

The service is not "in front" of MySQL, but is on the side of it so far as the app is concerned.

Post reply on HN