301–310 of 469 posts
You've gotta tip your hat to Sun, despite the OO crap and XML-itis the underlying ideas were - and are - sane.
Now, just as ejb was rightfully considered massive overkill for all but the most large-scale applications, just as simple servlet based (later on controller, service, persistence designs) we often more than enough to get the job done, same applies to uServices.
I have recently been thinking about doing microservices by simply having a shared MySql DB that all modules connect to. So for example one developer can create the backend and one developer can create the frontend. The codebases can be completely independent. One could use PHP+Laravel and the other one PHP+Symfony for example. Frontend and backend would live on their own servers. And simply having the IP, login and P…
I've been down that road. Eventually it turns into a nightmare because evolving your DB means making simultaneous changes to multiple applications. "We think we can drop this column, someone figure out which of our eight apps using this DB might be using it still" I much prefer putting a single service layer in front of the DB that speaks thrift or protobuf and letting all clients interface with that instead. Evolvin…
Yeah, agree. This is exactly how big companies end up doing, very interesting approach!
It's inevitable that someone has to invent another abstraction, but it does pay the bill. One thing you can do with this single layer is to control the flood gate.
I remember hearing this from a talk given by someone working at Instagram: as the team grew larger, cooperating on DB changes became problematic. So Instagram picked up Facebook's Tao.
These two statements don't reconcile: > Being both a developer and an operator is already tough (but critical to build good software) > Yes, with effective automation, monitoring, orchestration and so on, this is all possible This argues for having a separate ops team to handle the complexity centrally (e.g. operating a kubernetes cluster, providing standard building blocks), so developers can concentrate on their se…
However, I don't actually think a central team is always necessarily the best way to handle the cluster, it depends on the setup in the org and how many people are using it. In some cases it might be best to have the team who use it most heavily handle it, and share their best developers with other teams on rotation. But there's lots of options here. I agree the points are a bit unclear!
Earlier quoted context omitted.
The key to microservices is a framework and tooling around them to make them work for you. Release management, AuthN/AuthZ, compilation, composition, service lookup, etc. should all be "out-of-the-box" before microservices should ever be considered. Otherwise the O(n) gains you get in modularity turn into O(n) FML.
Would love to hear some peoples' thoughts on good contenders in each of these areas. What are some good boxes to get these out of in 2018?
It does have a single point of failure that is also a bottleneck (the router) and it's a lot of work to load balance / failover it.
But other than that, it provides a fantastic way to make microservices:
- anything is a client of the crossbar router. Your server code is a client. The db can be a client. The web page JS code is a client. And they all talk to each others.
- A client can expose any function to be called remotely by giving it a name. Another client can call this function by simply providing the name. Clients don't need to know each others. Routed RPC completly decoupled the clients from each others.
- A client can subscribe to a topic at any time and be notified when anothe client publishes a message on that topic. Quick, easy and powerful PUB/SUB with wildcard.
- A client can be written in JS (node and browser), Python, C#, Java, PHP, etc. Clients from other languages can talk to each others transparently. They all just receive JSON/msgpack/whatever and error are propagated and turn into the native language error handling mechanisme transparently.
- The API is pretty simple. If you tried SOAP or CORBA before, this is way, way, way simpler. It feels more like using redis.
- It uses websockets, which mean it works anywhere HTTP works. Including a web page or behind a NAT or with TLS.
- Everything is async. The router handles 6000 msg / s to 1000 clients on a small Raspberry Pi.
- The router can act as a process manager and even a static / wsgi web server.
- you can load balance any clients, hot swap them, ban them, get meta event about the whole system, etc.
- you can asign clients permissions, authentications, etc. Sessions are attached to each connexion, so clients knows what other clients can do.
Biggest issue with microservices: "Microservices can be monoliths in disguise" -- I'd omit the can and say 99% of the time are . It's not a microservice if you have API dependencies. It's (probably) not a microservice if you access a global data store. A microservice should generally not have side effects. Microservices are supposed to be great not just because of the ease of deployment, but it's also supposed to mak…
Earlier quoted context omitted.
That may be a bad analogy, considering painting your own apartment is something a great deal of people do, and often with good success. My parents (not in any way experts on that field) painted their entire house themselves, except for two rooms that were painted by a professional painter, and the professional painter left much worse corners than my parents. This was the paid-for result (ignore the dark corner at the…
Aside from the "sample size of one"-issue: how much time did your parents take, and how much effort and preparation time, and how did that compare to the time and effort the professional needed? It could be a trade-off issue there. Also, note that this thread started with a comment about how many developers don't appreciate what the actual hard problems in their own line of work are, and how that produces poor result…
Painting isn't a profession with a particularly high return on experience. You hire a painter for comparative-advantage reasons. It may take him just as long as it takes you, and several thousand dollars more, but chances are you can make more than several thousand dollars in the time you would've spent painting your house. (If you can't, you may want to re-think hiring a professional, and instead go into business as a painter yourself...)
Earlier quoted context omitted.
I call this the painting problem. Painting the walls of a room seems easy to an amateur: You just buy a few gallons at Home Depot and slap it on. But a professional knows that prep, trim, and cleanup are 80% of the job and they take skill. Anybody can slap paint onto the middle of a wall. What's difficult and time-consuming are making the edges sharp and keeping paint off the damn carpet.
My experience is a careful amateur painter is 1000% better than an average professional. Professionals are certainly a lot faster, but if you look carefully at their work it is in the main very shoddy. If you want a good result don’t skimp on the tools. Buy good quality brushes, rollers, filler, throws and paint. Also buy an edger to cut in the walls and ceilings. One final tip buy some of the disposable plastic line…
I think "microservices" is so appealing because so many Developers love the idea of tearing down the "old" (written >12 months ago), "crusty" (using a language they don't like/isn't in vogue) and "bloated" (using a pattern/model they don't agree with) "monolith" and turning it into a swarm of microservices. As an Infrastructure guy, the pattern I've seen time and time again is Developers thinking the previous generat…
I really think microservices are a process win not a technical win. It's easier and better to have 5 teams of 10 managing 5 services. Then having one team of 50 managing a one super service. When I see a team of 7 deciding to go with microservices for a new project I know they're gonna be in for a world of unnecessary pain.
I chose this approach because the developers who were already there were relatively new to C#, and I knew we were going to have to ramp up contractors relatively fast.
Our dev ops process revolves around creating build and release processes by simply cloning an existing build and release pipeline in Visual Studio Team Services - the hosted version of Team Foundation Services - and changing a variable. Every service is a separate repo. Each dev is responsible for releasing their own service.
The advantages:
1. All green field development for a new dev. They always start with an empty repo when creating a new service.
2. Maintenance is easier. You know going in all you have to do is use a few documented Postman calls to run your program if you need to make changes. Also, it's easy to see what the program does and if you make a mistake, it doesn't affect too many other people if you keep the interface the same.
3. The release process is fast. Once we get the necessary approvals, we can log on to VSTS from anywhere and press a button.
4. Bad code doesn't infest the entire system. The permanent junior employees are getting better by the month and we are all learning what works and doesn't work as we build out the system. Each service is taking our lessons learned into account. We aren't forced to keep living with bad decisions we made earlier and building on top of it.
A microservice strategy only works if you have the support system around it.
In our case, an easy to use continuous integration, continuous deployment system (VSTS), easy configuration (Consul), service discovery and recovery (Consul with watches), automated unit and integration tests, a method to standardize cross cutting concerns
And finally, Hashicorp's Nomad has been a god send for orchestration. Our "services" are really just a bunch of apps. Nomad works with shell scripts, batch files, Docker containers, and raw executables. It was much easier to set up and configure than kubernetes.
Biggest issue with microservices: "Microservices can be monoliths in disguise" -- I'd omit the can and say 99% of the time are . It's not a microservice if you have API dependencies. It's (probably) not a microservice if you access a global data store. A microservice should generally not have side effects. Microservices are supposed to be great not just because of the ease of deployment, but it's also supposed to mak…
Do you have examples of micro services and good data warehouses working well side by side? Your point makes sense, but I keep hoping for a way to have One Data Source Truth working side by side with the services that across it.
I hate to burst your bubble, but you shouldn’t and can’t have truth working along side systems that access it. Data is messy and tends toward dishonesty. The only way to get clean truth for your organization is by thoughtfully applying rules, cleaning and filtering as you go. The more micro your architecture is, the more this is true. Because there is no way 20different teams are all going to have the same understanding of the business rules around what constitutes good, clean input data. Even if your company is very clear and well-documented about business and data rules, if you hand the same spec sheet to 20 different teams, you are going to get 20 variations on that spec.
The only way to get usable data that can be agreed upon by an entire company (or even business unit) is by separating your truth from your transactional data. That’s kind of the definitional of a data warehouse.
If you let your transactional systems access and update data directly in your warehouse, you are in for a universe of pain.
Earlier quoted context omitted.
I agree that it's _easier_ for teams to have their own little fiefdoms, but not necessarily _better_. Shipping the org-chart is often a symptom of a leadership problem. When natural service boundaries exist, good leadership may choose to ship the org-chart, but too often extrinsic factors such as the arrangement of dev's desks dictates the architecture.
> ...but too often extrinsic factors such as the arrangement of dev's desks dictates the architecture. hahaha Do you know if there are some well chronicled cases of this happening? I find it very believable, but you know, would love to read something "actual".
A kinda-random example off the top of my head of "shipping the org chart" would be the historical gaps between Windows, Development, and Office at Microsoft. Ie Office is getting a new ribbon, but no development can't supply those icons or any components because they're an "office thing", or the internal API/VBA battles along the same lines.
On the opposite side, as reported in the press, Amazon has used this effect to create manageable teams and build up their own SOA: the two-pizza rule for teams means that teams can only really make targeted self-contained services. In this case Amazon worked backwards and re-structured their teams so that 'shipping their org chart' created the desired architecture.