131–140 of 782 posts
Before you go down the path of splitting your app up into microservices you should grok erlang/BEAM/OTP. Lots of thinking went into its creation that leads to highly reliable real time systems and at the very least some of the ideas can be lifted in informing how to best design things. (But really, you should probably just use it instead.)
I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries. Fanning out one big service in parallel with a matching scalable DB is by far the most sa…
You push data consistency concerns out of the database and between service boundaries. Gotta put that CS Masters to work somehow. Can't just sit here doing plumbing all day every day. Maybe there could be a software development corollary to the Politician's Syllogism[1], or even just a webdev one. 1. https://en.wikipedia.org/wiki/Politician%27s_syllogism
Microservices are an organizational and design choice that is intended to mirror the structure of the teams or engineers, the actual human beings doing this stuff. It seems like Segment didn’t really understand this at all, and instead decided to have seemingly arbitrary and rediculous service boundaries that had no relationship to the real world. See also people who create poor abstractions in their code and other s…
That's totally unnecessary and reflects poorly on you for having said so.
Not that I disagree with microservices easily going awry, the problem here seems to be traced to shared library code. Each microservice should be as standalone as possible. Your contract with that service is the service contract. Not some shared library. As soon as you have shared library, you now have coordinated deployments. And that is just not fun and will cause problems. The trick here is that this does mean you…
There is commonality across those 140 services that they extracted into a shared library so they weren’t copying that commonality 140 times.
The more I read about the problems people have with microservices, the more I'm convinced they've never read about flow-based programming.
Microservices are an organizational and design choice that is intended to mirror the structure of the teams or engineers, the actual human beings doing this stuff. It seems like Segment didn’t really understand this at all, and instead decided to have seemingly arbitrary and rediculous service boundaries that had no relationship to the real world. See also people who create poor abstractions in their code and other s…
As a principal engineer at Segment with over 20 years’ experience at successful companies large and small, I can personally attest to the fact that our team is incredibly talented and thoughtful. We make our engineering decisions very carefully and are highly evidence driven. Our choices are highly pragmatic and we investigate and analyze the trade-offs whenever we make a significant change such as this. I’m proud of…
Microservices are an organizational and design choice that is intended to mirror the structure of the teams or engineers, the actual human beings doing this stuff. It seems like Segment didn’t really understand this at all, and instead decided to have seemingly arbitrary and rediculous service boundaries that had no relationship to the real world. See also people who create poor abstractions in their code and other s…
I don't they think are worse than anyone else. It takes courage to write about failures and micro services are a complex thing to get right.
Earlier quoted context omitted.
Not really. For example, it's easier to mock a microservice, than a module, for testing purposes. Let's say you have component A and component B, A depends on B (dependency implemented via runtime sync or async call), B is computationally intensive or has certain requirements on resources that make it harder or impossible to test on developer's machine. You may want to test only A: with monolithic architecture you'll…
Mocking modules for unit testing has been a solved problem for decades in almost every language.
Microservices are an organizational and design choice that is intended to mirror the structure of the teams or engineers, the actual human beings doing this stuff. It seems like Segment didn’t really understand this at all, and instead decided to have seemingly arbitrary and rediculous service boundaries that had no relationship to the real world. See also people who create poor abstractions in their code and other s…
As a principal engineer at Segment with over 20 years’ experience at successful companies large and small, I can personally attest to the fact that our team is incredibly talented and thoughtful. We make our engineering decisions very carefully and are highly evidence driven. Our choices are highly pragmatic and we investigate and analyze the trade-offs whenever we make a significant change such as this. I’m proud of…
I'm not sure what they've been left with is a monolith after all. I would say they just have a new service, which is the size of what they should have originally attempted before splitting. In particular, as to their original problem, the shared library seems to be the main source of pain and that isn't technically solved by a monolith, along with not following the basic rule of services "put together first, split la…
Agreed. Reading about their setup and comparing with some truly large scale services I work with, I'm left with the idea that Segment's service is roughly the size of one microservice on our end. Perhaps the takeaway is don't go overboard with fragmenting services when they conceptually fulfill the same business role. And regardless of the architecture of the system, there are hard state problems to deal with in asso…
Some rules of thumb I just came up with:
Number of repos should not exceed number of developers.
Number of tests divided by number of developers should be at least 100.
Number of lines of code divided by number of repos should be at least 5000.
Your tests should not run faster than the time it takes to read this sentence.
A single person should not be able to memorize the entire contents of a single repo, unless that person is Rain Man.
Microservices are an organizational and design choice that is intended to mirror the structure of the teams or engineers, the actual human beings doing this stuff. It seems like Segment didn’t really understand this at all, and instead decided to have seemingly arbitrary and rediculous service boundaries that had no relationship to the real world. See also people who create poor abstractions in their code and other s…
Nothing you said addresses the issues they mentioned: > In early 2017 we reached a tipping point with a core piece of Segment’s product. It seemed as if we were falling from the microservices tree, hitting every branch on the way down. Instead of enabling us to move faster, the small team found themselves mired in exploding complexity. Essential benefits of this architecture became burdens. As our velocity plummeted,…