Earlier quoted context omitted.
Single writer multiple readers, ideally with permissions to enforce, is a useful hybrid. Enforce going through the front door if you want side-effects. Reporting and ad-hoc data enrichment can be painful to materialize otherwise. When you have multiple bits of code responsible for writing the same columns, maintaining global invariants becomes much harder. I can still see rationale for exceptions to the rule, e.g. st…
Caveat: I am really not qualified to discuss the nuances (because I have never used microservices so the little I know is based on reading a bit on those here and on other online forums). "Single writer multiple readers", yes, this is what I would probably use, but yet again, wasn't the "promise" of Microservices being able to work in total isolation? If I have one table (e.g. "Customer") which is written by one spec…
Don't start with microservices – monoliths are your friend
221–230 of 468 posts
Re: Don't start with microservices – monoliths are your friend
#222Unless you have a strong technical or organizational reason to use microservices, using microservices is just more work to achieve the same results. Organizational reason would be multiple people/teams who don't want or can't talk much to each other, so they develop pieces of a larger system as relatively independent projects, with clear API and responsibility boundaries. Frontend/backend style web development is an…
The monolith where most API endpoints are instant and use constant memory, but some use much more memory and can be slower... is tough.Like if you just give a bunch of memory to each process now you're overprovisioning and if you try to be strict you run into quality of service issues.
If you split out homogenous API endpoints into various groups you now have well behaved processes that are each acting similarly. One process could be very small, another could be much larger (but handle only one kind of request), etc...
of course the problem with standard microservice-y stuff is now you gotta have N different applications be able to speak your stack. The idea of a monolith with feature sets is tempting... but also can negate chunks of microservice advantages.
Ultimately the microservice-y "everything is an API" can work well even as a monolith, and you would then have the flexibility to improve things operationally later.
Re: Don't start with microservices – monoliths are your friend
#223Earlier quoted context omitted.
I wholeheartedly agree on most parts, with DevOps being the exception. Devs only focusing on developing/writing source code is certainly not the way to produce sustainable, high quality software- at least in my opinion...
I don't know. I totally agree on the premise that developers should be involved in ops as well. But I'm a developer by heart and my heart aches whenever I see what we devs have wrought in the ops space. (Insert joke about the CNCF technology landscape map.) There's just so many tech/tools/etc. involved that just reasonably "doing the ops stuff" on the side seems way unrealistic. Sometimes I feel that all we've accomp…
> But I'm a developer by heart and my heart aches whenever I see what we devs have wrought in the ops space.
At the same time, I am conflicted. I don't care for the toss-it-over-the-wall approach that used to be the norm, but I also don't like having dev's have to take more on than they are capable.
In an ideal environment, I would like to see a crashing together of developers and operations people on teams. What I mean is that for a given team you have several developers and one or two ops folks. This way there is less of an us vs them sentiment and teams can be held accountable for their solutions from ideation to running in production. Finally while it's not the sole responsibility of the dev's to manage their DevOps stuff, they will have more knowledge of how it works, and get to put input and ideas to it.
Re: Don't start with microservices – monoliths are your friend
#224Earlier quoted context omitted.
> no good reason How about deployment speed? If I’ve got a microservice collecting events off a queue and writing a csv out to S3 on a schedule, it’s really nice to be able to add a column and deploy in minutes without having to rebuild and deploy a giant monolith. It also allows for fine grained permissions: that service can only read from that specific queue and write to that specific bucket. People throw around “d…
Distributed monolith can mean different things. I worked at a place with ~10 services sharing the same DB. It was awful
Re: Don't start with microservices – monoliths are your friend
#225Earlier quoted context omitted.
Not so much snarky as a commentary on the diligence. Yes, if there is mission critical software that affects millions of people, do all that stuff. If your pdf -> stardust service is more a convenience then don't spend hundreds of thousands on infrastructure.
> ...then don't spend hundreds of thousands on infrastructure. I find it curious how we went from doing the basics of software development that would minimize risks and be helpful to almost any project out there to this. To clarify, i agree with the point that you'll need to prioritize different components based on what matters the most, but i don't think that you can't have a common standard set of tools and practic…
The main point is, that one should never spend a "a few days to a week" to implement a feature that at best i useless and at worst is detrimental to the service stood up.
Implement auth, if it is needed, implement monitoring, CI, CD, dependency monitoring, testing, everything, if it is needed.
But don't implement it as dogmatic consequences of doing software development.
And regarding the spend: one week worth of work could be USD 8k. So just the initial implementation of your JWT based authentication system is 4% into the "hundreds of thousands of dollars". Then you need to factor in the extra complexity on maintenance and before you know it we do not talk about hundreds of thousands of dollars but millions...
Re: Don't start with microservices – monoliths are your friend
#226Maybe I'm too old, but I don't even want to have to worry about all that. I think in terms of functions and I don't care if they are being called remotely or local. That was the promise back in the day of J2EE, and it seems to me Microservices are just a rehash of that same promise. Which never really worked out with J2EE - it was mostly invented to sell big servers and expensive consultants, which is how Sun made mo…
DevOps is a sad story both for devs and for ops. It was supposed to treat operations as a SW problem thus take away the toll and draw devs in. In reality for most places it either means that devs also do pipelines and operations or that operations were rebranded and are using "DevOps" tooling to do operations.
Re: Don't start with microservices – monoliths are your friend
#227Re: Don't start with microservices – monoliths are your friend
#228There is a lot of talk about monoliths vs microservices lately.. I just want to throw into the ring that you can do both at the same time. easily. And noone is going to kill you for it either. maybe we are getting caught up in sematics because its christmas, but "monorepo/monolith/microservices/etc" is -just- the way you organize your code. Developing a montolith for years but now you have written a 15 line golang ht…
> Developing a montolith for years but now you have written a 15 line golang http api that converts pdfs to stardust and put it into on a dedicted server in your office? welp thats a microservice. But the 15 lines of Golang are not just 15 lines of Golang in production. You need: - auth? Who can talk to your service? Perhaps ip whitelisting? - monitoring? How do you know if you service is up and running? If it's down…
So in the k8s world:
- auth: service meshes, network policies, ...
- monitoring: tons of tooling there to streamline that
- deploy: this at scale is trickier than you'd think, many seem to assume k8s on it's own here is the magic dust they need. But GitOps with ArgoCD + helm has worked pretty well at scale in my experience.
- Security is a CI problem, and you have that with every single language, not just Go. See Log4j.
Kubernetes is my bread & butter, but I do realise this has way too much overhead for small applications. However, once you reach a certain scale, it solves many of the really really hard problems by streamlining how you look at applications from an infrastructure and deployment side of things. But yes - you need dedicated people who understand k8s and know what the hell they're doing - and that's in my experience a challenge on it's own.
Let's also dispel a myth that k8s is only suitable for microservices. I have clients that are running completely separate monolith applications on k8s, but enough of those that managing them 'the old way' became very challenging, and moving these to k8s in the end simplified thing. But getting there was a very painful process.
Re: Don't start with microservices – monoliths are your friend
#229I wonder why no one ever talks about architectures in the middle between those two - modular monoliths. The point in time where you're splitting your codebase up in modules (or maybe are a proponent of hexagonal architecture and have designed it that way from the beginning), leading to being able to put functionality behind feature flags. That way, you can still run it either as a single instance monolith, or a set o…
Micro services were not an answer to monolith being bad. Something somewhere went really wrong with people's understanding and there is bunch of totally wrong ideas.
That is also maybe because a lot of people did not knew they were supposed to make modules in their code and loads of monoliths ended up being spaghetti code just like now lots of micro service architectures end up with everything depending on everything.
Re: Don't start with microservices – monoliths are your friend
#230Earlier quoted context omitted.
I've argued for a long time that microservices should grow out of a monolith like an amoeba. The monolith grows until there is a clear section that can be carved off. Often the first section is security/auth, but from there it's going to be application specific. A modulith could be just another step in the carve up process. But, there is no right answer here. Application domain, team size, team experience, etc... all…
In my experience with enterprise software one of the things that cause most trouble is premature modularization (sibling to the famous premature optimization). Just like I can't understand how people can come up with the right tests before the code in TDD, I can't understand how people can come up with the right microservices before they start developing the solution.
Likewise, with a microservice architecture, you have requirements that define a set of data C that must be available for an application that provides get/post/put/delete/events in a set B to your service over a transport protocol. You need to provide access to this data via the same transport protocol, transform the input protocol to a specified output protocol.
You also have operational concerns, like logging that takes messages in a set C and stores them. And monitoring, and authorization, etc. These are present in every request/response cycle.
So, you now split the application into request/response services across the routing boundary-> 1 route = 1 backing model. That service can call other apis as needed. And that's it. It's not hard. It's not even module-level split depolyment. It's function-level deployment in most serverless architectures that is recommended because it offers the most isolation, while combining services makes deployment easier, that's mostly a case of splitting deployment across several deployment templates that are all alike and can be managed as sets by deployment technologies like Cloudformation and Terraform [1].
You can also think of boundaries like this: services in any SOA are just like software modules in any program - they should obey open/closed and have strong cohesion [2] to belong in a singular deployment service.
Then you measure and monitor. If two services always scale together, and mutually call each other, it's likely that they are actually one module and you won't effect cohesion by deploying them as a single service to replace the two existing ones. easing ops overhead.
Not deploying and running as a monolith doesn't mean not putting the code to be run into the same scm/multiproject build as a monorepo for easy shared cross-service message schema refactoring, dependency management, and version ingredient. That comes with its own set of problems -- service projects within the repo that do not change or use new portions of the comm message message schema shouldn't redeploy with new shared artifact dependencies; basically everything should still deploy incrementally and independently, scaling it is hard (see Alphabet/Google's or Twitter's monorepo management practices, for example); but there seems to be an extra scale rank beyond Enterprise size that applies to, it's very unlikely you are in that category, and if you are you'll know it immediately.
We like to market microservice architecture as an engineering concern. But it's really about ops costs in the end. Lambdas for services that aren't constantly active tend to cost less than containers/vps/compute instances.
1: http://serverless.com//blog/serverless-architecture-code-pat...
2: http://www.cs.sjsu.edu/faculty/pearce/modules/lectures/ood/m...