3 full time engineers. No wonder they crashed and burned with microservices. Even a slightly distributed system, what I call miniservices, with an api and login server and half a dozen regular apps on top is way too much for such a small team. In a similar, but much smaller setup, I estimate at one point our two person team was spending as much as 25% of their time on cross-service (between the app and api / login server) concerns that would simply not exist with a monolith or something not distributed. That's 3 months out of every year of development time wasted on concerns that shouldn't even exist.
Goodbye Microservices: From 100s of problem children to 1 superstar
521–530 of 782 posts
Re: Goodbye Microservices: From 100s of problem children to 1 superstar
#522 - All teams will henceforth expose their data and functionality through service interfaces.
- Teams must communicate with each other through these interfaces.
- There will be no other form of inter-process communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
- It doesn’t matter what technology they use.
- All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
- Anyone who doesn’t do this will be fired. Thank you; have a nice day!
Re: Goodbye Microservices: From 100s of problem children to 1 superstar
#523Earlier quoted context omitted.
Right, but the thing that makes Linux actually useful isn't really the kernel is it? I would say what makes it useful is all the various small, targeted programs (some might call them microservices) it lets you interact with to solve real world problems. If Linux tried to be an entire computing system all in one code base, (sed, vim, grep, top, etc., etc.) what do you think that would look like code base/maintainabil…
One example I'm familiar with that sounds like microservices is the Robot Operating System (ROS). At its heart it's just a framework for pub/sub over IP, it just happens to be targeted towards robotics. A ROS system comprises of 'nodes' for each logical operation e.g. image acquisition -> camera calibration -> analysis -> output. The system is defined by a graph of these nodes, since you can pipe messages wherever th…
IMO, the main benefit of ROS is the message definitions. Having exactly one definition of "point cloud" that everybody accepts means everyone's code will be compatible. That isn't normally the case in library ecosystems. If ROS was replaced by I think we'd get 90% of the benefit.
Re: Goodbye Microservices: From 100s of problem children to 1 superstar
#524Earlier quoted context omitted.
This means either your contract agreement between services is not language agonistic or someone did not do their job when Interface schema was changed during reviews. Just like any other software, interfaces live in separate repos e.g protoplasm/ for proto-buf definitions. avrobber/ for avro and so on so forth. So, any change to protoplasm/ triggers automated tests on all other services irrespective of boundaries.
ok. But _much_ easier to have compile time checking between projects in the same language/repo. Not saying this approach is wrong, but it's a little like riding a unicycle when a perfectly good bicycle is sitting right there.
I think I did not present this correctly, I apologize for that.
What I mean this this. Any IDL interfaces live at their functional boundaries. e.g all proto definitions shared internally by java services live under jumanji/proto/, all proto definitions shared internally bu Go service live under goat/proto/ and so on so forth.
But anything which is shared in a public manner e.g any proto definitions between Java and C live outside either of these repos in top level common_proto/ repo. The build system is configured to take care of pulling the correct proto version from common_proto repo.
We use maven for this, but nothing much we can do about that. Build systems are ugly and we have to live with maven in near future.
EDIT: Oh that said, stop using required field in protobufs if you are stuck using proto2 like us, whoever decided to add required field to proto2 grammar made a mess. Things change and deprecate repeatedly, gazillions of required fields are annoying and recipe for disaster.
Re: Goodbye Microservices: From 100s of problem children to 1 superstar
#525This is a classic case of not understanding micro services and trying to fit a problem around a tool. At work, we have close to ~50 services(no one calls them microservices), but they do not suffer from this brittleness. We segregate our services based on languages. So, all C services go under coco/ , all Java services go under jumanji/ , all go services go under goat/ , all JS services go under js/. This means, ever…
Can you be the Jack Doneghy to my Liz Lemon?
Please?
Re: Goodbye Microservices: From 100s of problem children to 1 superstar
#526Earlier quoted context omitted.
I think what was missed, in the article, is that the fundamental problem was centered around a shared architecture of destinations and shared code. You cannot possibly have every destination be a separate repo and then have the development lifecycle of your shared code be so active that it ultimately puts at risk the architecture of your entire organization. What makes shared code so perfect is having stability such…
> ... the fundamental problem was centered around a shared architecture of destinations and shared code. From an architectural perspective, there is absolutely no difference between a micro service and a library. The only real difference is in the dispatch mechanism. The problems around configuration management are the same problems we've had as programmers for decades. It's just that the people who are keen on micro…
> "should we implement ld.so for micro services? :-)"
Working exactly along those lines this week on multiple services exposed over REST APIs, I was wondering if tools exist to check compatibility between them. Said differently,
- I have a `swagger.yaml` for my service A managing chipmunks and it says endpoint `/chipmunks` supports an 'color' query parameter.
- In service B, I have a `handleToServiceA` that encapsulates calling A. Then say I write `const chipmunks = handleToServiceA.getChipmunks({'colour': 'blue'})`.
Are there (whatever the ecosystem) tools that would read serviceA's swagger.yaml, detect my error in serviceB (color -> colour) and report the issue at compile time rather than run time?
Re: Goodbye Microservices: From 100s of problem children to 1 superstar
#527Earlier quoted context omitted.
It's worse than that; it's my observation that most microservice architectures just ignore consistency altogether ("we don't need no stinking transactions!") and blindly follow the happy path. I've never quite understood why people think that taking software modules and separating them by a slow, unreliable network connection with tedious hand-wired REST processing should somehow make an architecture better. I think…
I've heard of even Unicorns throwing consistency out the window. Apparently Netflix has a bunch of "cleanup jobs" that comb the database for various inconsistencies that inevitably show up. You can't have consistent microservices without distributed transactions. If a service gets called, and inside that call, it calls 3 others, you need to have a roll back mechanism that handles any of them failing in any order. If…
Data consistency is one of those things that sounds like it matters but often doesn't. There's not much in Netflix's platform that screams data consistency is an ultra high priority. Any application that deploys active/active multi-region is by definition going to encounter scenarios where data loss is possible. There's just no way around CAP.
I'd venture a guess that most applications have all sorts of race conditions that could cause data corruption. The fact of the matter is that almost nobody notices or even cares.
Re: Goodbye Microservices: From 100s of problem children to 1 superstar
#528Re: Goodbye Microservices: From 100s of problem children to 1 superstar
#529Earlier quoted context omitted.
This cycle blankets every new technology as a misstep that we’ll revert back from. The real cycle is that we continually test new ideas and new approaches, learn their strengths and weakness, and ultimately keep what works and discard what doesn’t. Sure we miss the ball more than we connect. But take this example: We now have native JSON support since Postgres 9.2. So did we abandon the document model or learn more a…
FTP workflows are so extremely widely used in today's world that it's inconceivable you think it's behind us.
Maybe in frontenddev/small companies/small largely static websites FTP pervades, but I'd argue that most developers that deal with any real complexity probably use some kind of source control system to 'share' files rather than some kind of FTP program. And I'd be very surprised if relative FTP usage hadn't decreased significantly over the last 3-4 years (But this is all quite anecdotal, and I could be wrong).
Something that does surprise me is that Panic's Coda and Transmit apps still seem quite successful, so maybe my perception is out of whack.
Re: Goodbye Microservices: From 100s of problem children to 1 superstar
#530Earlier quoted context omitted.
The simple fact is mongoDB fails to have sufficient durability guarantees for business data when compared to a traditional RDBMS. That is one of the reasons why its so fast in comparison. A power loss event during data writes can and will lead to data loss. https://www.mongodb.com/blog/post/what-about-durability
Sounds like a "right tool for the job" situation. I thankfully don't have to worry about power loss (large company, teams dedicated to critical systems infrastructure in a big way), and a cron job can handle any archival concerns in the projects I'm running here. It does sound like they've improved much since 2011. Anyway thanks for the insight
MongoDB 4.x also brings transactions to the table, which further reduces the vulnerability to inconsistent data. Data loss is far less of a problem than inconsistency due to partially completed operations and data damage.