Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye Microservices: From 100s of problem children to 1 superstar

segment.com

Post reply on HN
Print view View original

121–130 of 782 posts

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#121

Post by fastball »

halayli wrote: post #109
iamleppert wrote: post #75

Microservices are an organizational and design choice that is intended to mirror the structure of the teams or engineers, the actual human beings doing this stuff. It seems like Segment didn’t really understand this at all, and instead decided to have seemingly arbitrary and rediculous service boundaries that had no relationship to the real world. See also people who create poor abstractions in their code and other s…

Nothing you said addresses the issues they mentioned: > In early 2017 we reached a tipping point with a core piece of Segment’s product. It seemed as if we were falling from the microservices tree, hitting every branch on the way down. Instead of enabling us to move faster, the small team found themselves mired in exploding complexity. Essential benefits of this architecture became burdens. As our velocity plummeted,…

iamleppert does address that. As they said, microservices are supposed to mirror the structure of your team(s).

A single team should never be maintaining 140 different microservices. That's not a failing of the microservices architecture, that is a failing of massively abusing the microservices architecture. Arguably, a small team should never be managing more than 10 services (totally arbitrary number, but it seems like the upper limit to what a human can focus on).

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#122

Post by riknos314 »

Working at a large tech company, we tend to approach microservice division as information authority division.

If the data doesn't need to exist in the same database, put it in a new service with a separate database (or at least completely independent tables).

Ideally there is no shared code between services (and there shouldn't need to be, because each service owns completely disparate data), so the only coupling is the API definitions.

Each service is free to use whatever internal architecture they see fit as long as they honor the API definitions they provide to their dependent services.

In the case outlined in the article, the fact that each microservice had similar enough concerns to all use some common libraries and the same database makes me doubt that this should ever have been built as microservices to begin with.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#123

Post by acroback »

This is a classic case of not understanding micro services and trying to fit a problem around a tool.

At work, we have close to ~50 services(no one calls them microservices), but they do not suffer from this brittleness. We segregate our services based on languages. So, all C services go under coco/ , all Java services go under jumanji/ , all go services go under goat/ , all JS services go under js/. This means, everytime you touch something under a repo, it affects everyone. You are forced to use existing code or improve it, or you risk breaking code for everyone else. What does this solve? This solves the fundamental problem a lot of leetcode/hackerrank monkeys miss, programming is a Social activity it is not a go into a cave and come out with a perfect solution in a month activity. More interaction among developers means Engineers are forced to account for trade offs. Software Engineering in its entirety is all about trade offs, unlike theoretical Comp Science.

Anyway, this helps because as Engineers we must respect and account for other Engineers decisions. This methods helps tremendously to do this. No one complains, everyone who wants 1000 more microservices usually turns out to be a code monkey entangled in new fad, or who doesn't want to work with other Engineers.

You want to use rust? There is a repo named fe2O3/, go on. Accountability and responsibility is on your shoulders now.

If you think about it, an Engineer is tied to his tools, why not segregate repos at language level instead of some arbitrary boundary no one knows about in a dynamic ecosystem?

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#125

Post by sulam »

halayli wrote: post #109
iamleppert wrote: post #75

Microservices are an organizational and design choice that is intended to mirror the structure of the teams or engineers, the actual human beings doing this stuff. It seems like Segment didn’t really understand this at all, and instead decided to have seemingly arbitrary and rediculous service boundaries that had no relationship to the real world. See also people who create poor abstractions in their code and other s…

Nothing you said addresses the issues they mentioned: > In early 2017 we reached a tipping point with a core piece of Segment’s product. It seemed as if we were falling from the microservices tree, hitting every branch on the way down. Instead of enabling us to move faster, the small team found themselves mired in exploding complexity. Essential benefits of this architecture became burdens. As our velocity plummeted,…

The important part here, that the parent addressed, is the statement "small team", as an organizational structure distinct from, say, a "large team" which has the opportunity to experience Conway's law.

It sounds like Segment very much prematurely optimized. Microservices is probably not a great approach below 20 developers, and becomes necessary past 100, at least in my experience.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#126

Post by johannes1234321 »

thermodynthrway wrote: post #3

I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries. Fanning out one big service in parallel with a matching scalable DB is by far the most sa…

The kernel can't be categorized fully as monolith: Many kernel subsystems are developed individually, by different teams and merged into the main branch by subsystem.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#127

Post by nickthemagicman »

hinkley wrote: post #42
thermodynthrway wrote: post #3

I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries. Fanning out one big service in parallel with a matching scalable DB is by far the most sa…

People look at microservices as a solution to the Big Ball of Mud, and then confuse "a solution" with "the solution". You really do have to modularize. In some languages, you can even use separate compilation units for separate modules to enforce the separation. You can do all of that but get simultaneous deployment, which cuts out whole classes of integration nightmares.

It seems like a Big Ball of mud is much more difficult to deal with than a bunch of small balls of mud just in general?

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#128

Post by fastball »

apeace wrote: post #107
NickBusey wrote: post #72

Earlier quoted context omitted.

Right, but the thing that makes Linux actually useful isn't really the kernel is it? I would say what makes it useful is all the various small, targeted programs (some might call them microservices) it lets you interact with to solve real world problems. If Linux tried to be an entire computing system all in one code base, (sed, vim, grep, top, etc., etc.) what do you think that would look like code base/maintainabil…

My understanding is that OpenBSD is constructed this way, and I have also heard that their code is well-organized and easy to follow. There is the "base" system which is the OS itself and common packages (all the ones you mentioned), then there is the "ports" repo which contains many open-source applications with patches to make them work with OpenBSD. Here is a Github mirror of their repos: https://github.com/openbs…

"packages" and "ports" are both collections of "microservices" in your analogy, so you can't really assert the whole thing is a monolith.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#129

Post by otterley »

iamleppert wrote: post #75

Microservices are an organizational and design choice that is intended to mirror the structure of the teams or engineers, the actual human beings doing this stuff. It seems like Segment didn’t really understand this at all, and instead decided to have seemingly arbitrary and rediculous service boundaries that had no relationship to the real world. See also people who create poor abstractions in their code and other s…

As a principal engineer at Segment with over 20 years’ experience at successful companies large and small, I can personally attest to the fact that our team is incredibly talented and thoughtful. We make our engineering decisions very carefully and are highly evidence driven. Our choices are highly pragmatic and we investigate and analyze the trade-offs whenever we make a significant change such as this. I’m proud of what we’re doing.

You can quibble with our implementations, but at the end of the day, the proof of the engineering is in the working. Our system works even better than before and our customers appreciate it!

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#130

Post by rhizome »

thermodynthrway wrote: post #3

I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries. Fanning out one big service in parallel with a matching scalable DB is by far the most sa…

You push data consistency concerns out of the database and between service boundaries.

Gotta put that CS Masters to work somehow. Can't just sit here doing plumbing all day every day.

Maybe there could be a software development corollary to the Politician's Syllogism[1], or even just a webdev one.

1. https://en.wikipedia.org/wiki/Politician%27s_syllogism

Post reply on HN