471–480 of 671 posts
Answer is no. It is about where the shoe fits. If you become too heavily dependent on modules you risk module incompatibility due to version changes. If you are not the maintainer of your dependent module you hold a lot of risk. You don't get that with microservices. If you focus too much on microservices you introduce virtualized bloat that adds too much complexity and complexities are bad. Modules are like someone…
Earlier quoted context omitted.
I just want to point out that for the second problem (scalability of CPU/memory/io), microservices almost always make things worse. Making an RPC necessarily implies serialization and deserialization of data, and nearly always also means sending data over a socket. Plus the fact that most services have some constant overhead of the footprint to run the RPC code and other things (healthchecking, stats collection, etc.…
Microservices are less efficient , but are still more scalable . Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers. But this only matters at Facebook scale. I think most devs would be…
Little big concept or mono-micro people like the call it is where it is at. Spending too much time making a complex environment benefits noone. Spending too much time making a complex application benefits noone.
Breaking down monoliths into purposed tasks and then creating pod groups small containers is where it is at. Managing a fleet on kubernetes is easier than managing one in some configuration management stack. Be it puppet or salt. You have too many dependencies. Only your core infrastructure kubernetes should be a product of the configuration management.
Earlier quoted context omitted.
# Microservices I think at the root this idea that Microservices provide this technical solution to a social issue, coordinating teams at scale, hides the driving motivation. Why can't the teams work on the same codebase and the same database? (again note my edited in disclaimer in the GP post that maybe at certain rare scales it's necessary). What is a well defined specification and why is it only a REST/proto/whate…
> Why can't the teams work on the same codebase and the same database? The same database is tricky because one team might want to change the schema, breaking another team's work. You then need to have meetings to discuss the changes that will work for all parties and you've broken "communicate only by contract", and therefore no longer doing microservices. If you can ensure that there is no way for teams to trample o…
Agreed the same database gets difficult but I guess, where two things need to change dependently there's coordination overhead, that's inescapable. You can in general use e.g. schemas to keep domains fairly separate. Whether it's your HTTP API interface changing or the change needs to be coordinated in the same process I don't think you have a way to avoid these meetings. (or like the 3rd party team I work with you just change your HTTP API and break production without telling us :shrug:).
I guess my point is languages like Java and C# are much maligned for their "boilerplate" and it doesn't let 10x genius programmers go brrrrrr etc. But that boilerplate serves a purpose. Projects (in C#, I can't speak to Java but I believe it has a similar concept) compile to separate .dll's, or .so's. In general a project will publicly expose interfaces, i.e. contracts, and keep its internals encapsulated. Crucially projects all live in the same codebase in the same monolith, they're folder/directory level separation, but on steroids. There are tricks with attributes and reflection that can break this separation but with the additional constraint that dependencies cannot be circular you're forced to make decisions about where exactly things live. This is where the dynamic crowd throw up their hands and say "there are so many layers of pointless abstraction!". I'd agree these languages tend to too many layers but the layers provide the foundation which can support a project growing from 1-2 devs all the way through to 50+.
I'm maybe finally becoming a grumpy old programmer and I'm glad much of the consensus is swinging away from the no-types, no schema, no rules position of the past decade. People forgot Chesterton's fence and decided the layers didn't provide anything, then they added them back, as network boundaries, which was indescribably worse and more of a headache.
# Testing
I guess it's a semantics debate really. I think the red-green-refactor loop of TDD doesn't bring much. I've never seen it applied well on a statically typed codebase and where I've seen advocates try and apply it I've seen them tunnel-vision to suboptimal outcomes in order to fulfil the ceremonial needs of the process.
I think a few tests that take place at the project/dll/so boundary and use as much of the real dependencies as possible are far preferable to high coverage. This is probably a no-true-Scotsman area of debate. Maybe TDD has been done properly and productively by some groups, I've never seen it and no one I've met ever has (check out my confirmation bias!).
Earlier quoted context omitted.
But, I am trying to feel my way towards the idea that that was true when managers arranged systems for workers to follow , and then we came along to automate the current process. But if we have a system where the workers are the CPUs, then the people doing the managing are the coders. The point ebing is that if workers are CPUs and coders are managers, then why worry about how the managers of the coders are arranged.…
The systems are not only organized on the programmers managers, but their managers too, all the way up the chain.
I've been a couple levels up the chain at times in my career. When I've seen technical decisions at this level done well is when those people realize that their organizational decisions are inevitably technical decisions because of Conway's law. When the impact of Conway's law is not explicitly considered, things will often look random at the IC level. Too often, organizational leaders don't realize they are dictating technical outcomes with decisions that appear to just be about reporting structure.
An example is when a major new product feature comes along that is big enough to need multiple teams to build/own it for the long term. Where are those teams placed in the organization? Conway's law helps us see that a huge part of the future architecture is being determined by answering that question. Placing those new teams in an existing org mainly responsible for UX will result in a very different architecture than putting the new teams in an org mainly responsible for monetization. Can the teams be split across those orgs? Well, can you envision a reasonable architecture with an enforced divide along those lines? Without recognition of Conway's law, that last question might not be considered and the decision to split or not will be based on other factors.
Unfortunately that requires multidisciplinary thinking (technical architecture, resource allocation, communication structure, people development, ...). Such broad thinking is not always easy. When leadership starts neglecting one or more of these disciplines, more and more of the people doing the work on affected teams start perceiving the decisions as arbitrary and counter productive. Organizational churn (e.g., teams repeatedly changing location within the org over the course of a year or two) is often leadership looking at a different subset of these dimensions over time, yielding different "correct" answers.
Earlier quoted context omitted.
The right answer is that there is no right answer. You shouldn't divide your code based on arbitrary metric like size, you should divide it based on concepts/domains. If a particular domain gets big enough it probably means it contains sub-domains and can benefit from being divided too. But you cannot make that decision based on size alone.
Sure but no problem domain is not going to lead you to 10,000 single line files. Similarly it will likely lead to very few 10K line files. There will likely be a way to factor things into reasonable sized chunks. File sizes are not going to be that highly coupled to problem domain as there are multiple ways to solve the same problem.
The point is that a numeric upper bound on LoC is inherently subjective and pointless. Instead of measuring the right thing (concepts) you're measuring what's easy to measure (lines).
In fact, it usually makes things worse. I've seen it over and over: you have N tightly coupled classes in a single file which exceeds your LoC preference.
Instead of breaking the coupling you just move those classes into separate files. Boom, problem solved. Previously you had a mess, now you have a neat mess. Great success!
Earlier quoted context omitted.
Microservices are less efficient , but are still more scalable . Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers. But this only matters at Facebook scale. I think most devs would be…
You know, I don't really think microservices are fundamentally more scalable. Rather, they expose scaling issues more readily. When you have a giant monolith with the "load the world" endpoint, it can be tricky to pinpoint the the "load the world" endpoint (or, as is often the case, endpoint*s*) is what's causing issues. Instead, everyone just tends to think of it as "the x app having problems." When you bust the mon…
It depends on what you are scaling. I think microservices are fundamentally more scalable for deployment, since changes can be rolled out only to the services that changed, rather than everywhere. Unless your language and runtime support hot-loading individual modules at runtime.
Earlier quoted context omitted.
> Why can't the teams work on the same codebase and the same database? The same database is tricky because one team might want to change the schema, breaking another team's work. You then need to have meetings to discuss the changes that will work for all parties and you've broken "communicate only by contract", and therefore no longer doing microservices. If you can ensure that there is no way for teams to trample o…
# Microservices Agreed the same database gets difficult but I guess, where two things need to change dependently there's coordination overhead, that's inescapable. You can in general use e.g. schemas to keep domains fairly separate. Whether it's your HTTP API interface changing or the change needs to be coordinated in the same process I don't think you have a way to avoid these meetings. (or like the 3rd party team I…
Earlier quoted context omitted.
Why would using microservices reduce the chance of outages? If you break a microservice that is vital for the system, you are as screwed as with a monolyth.
Sure, but not all micro-services are vital. If your "email report" service has a memory leak (or many other noisy-neighbor issues) and is in a crash loop then that wont take down the "search service" or the "auth service", etc. Many other user paths will remain active and usable. It compartmentalizes risk.
So it seems like we’re trying to compensate bad design with microservices. It’s orthogonal IMO.
Earlier quoted context omitted.
> Why can't the teams work on the same codebase and the same database? The same database is tricky because one team might want to change the schema, breaking another team's work. You then need to have meetings to discuss the changes that will work for all parties and you've broken "communicate only by contract", and therefore no longer doing microservices. If you can ensure that there is no way for teams to trample o…
# Microservices Agreed the same database gets difficult but I guess, where two things need to change dependently there's coordination overhead, that's inescapable. You can in general use e.g. schemas to keep domains fairly separate. Whether it's your HTTP API interface changing or the change needs to be coordinated in the same process I don't think you have a way to avoid these meetings. (or like the 3rd party team I…
Interfaces don't change under microservices. Communication by contract enshrines a contract. You must ensure that your API does not break legacy users no matter what changes you want to make going forward. You have committed to behaviour forevermore once you submit the contract to other teams.
This may be another reason why IPC is often preferred over straight function calls as some languages, particularly those with static types, make extending functionality without breakage quite hard. An HTTP API can more easily resort to tricks to return different results to different callers.
> I think the red-green-refactor loop of TDD doesn't bring much.
You find little value in writing a specification for your work or you find little value in automatic validation that the program works according to your spec?
"Red-green-refactor" is a little more specific in that it says that you should write a spec only for the work you know you are going to work on in the short term, whereas TDD in general leaves room for things like writing the entire application's spec before getting down to business. I think this is most practical in the real world, generally speaking. Often you don't know what your entire application should do in the beginning, making writing a full spec unrealistic.
> I've never seen it applied well on a statically typed codebase
Interesting, as I have only ever seen specifications (be it TDD or "Word documents") written for work that is done in statically typed languages. In my experience, the dynamic programming languages tend to attract the cowboy coders who don't understand why other (future) developers need more complete documentation or why specs are important and have no care to see that those things are done.
Earlier quoted context omitted.
But, I am trying to feel my way towards the idea that that was true when managers arranged systems for workers to follow , and then we came along to automate the current process. But if we have a system where the workers are the CPUs, then the people doing the managing are the coders. The point ebing is that if workers are CPUs and coders are managers, then why worry about how the managers of the coders are arranged.…
Managers arranging work is _not_ why Conway's law is true though. I think it behooves one to look at the actual text of the "law": > Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. It doesn't matter WHO defines or does the work (the coders themselves in a team or the manager). It matters how communication is arrang…
Just to add it here, how evaluations and goals are set matter too. Those tend to be even more constrained by the formal organization than communication.