Earlier quoted context omitted.
It wouldn't work - it's trivial to add meaningless query parameters or anchors that would change the hash but still lead to the same content. And stripping that wouldn't work because some sites use them to route to content. What might work is hashing the text and outbound link content submitted pages of, and building something like a similarity index of text, metadata and a graph of links, but that would probably sti…
They could capture the canonical URL from the meta tags in the page. I don't think they do currently.
The Death of Microservice Madness in 2018
41–50 of 469 posts
Re: The Death of Microservice Madness in 2018
#42Earlier quoted context omitted.
I've been down that road. Eventually it turns into a nightmare because evolving your DB means making simultaneous changes to multiple applications. "We think we can drop this column, someone figure out which of our eight apps using this DB might be using it still" I much prefer putting a single service layer in front of the DB that speaks thrift or protobuf and letting all clients interface with that instead. Evolvin…
How does an additional layer make changes to the DB easier? If a column is dropped what will your layer do when a request for data from that column comes in? What if the request joins it with other columns from other tables?
In the end, it's mostly about enforcing contracts and making devops simpler.
Re: The Death of Microservice Madness in 2018
#43Biggest issue with microservices: "Microservices can be monoliths in disguise" -- I'd omit the can and say 99% of the time are . It's not a microservice if you have API dependencies. It's (probably) not a microservice if you access a global data store. A microservice should generally not have side effects. Microservices are supposed to be great not just because of the ease of deployment, but it's also supposed to mak…
Once when starting a new gig I inherited a "microservices" architecture. They were having performance problems and "needed" to migrate to microservices. They developed 12 seperate applications, all in the same repo, deployed independently it's own JVM. Of course if you were using microservices, you needed docker as well, so they had also developed a giant docker container containing all 12 microservices which they de…
Re: The Death of Microservice Madness in 2018
#44Earlier quoted context omitted.
How does an additional layer make changes to the DB easier? If a column is dropped what will your layer do when a request for data from that column comes in? What if the request joins it with other columns from other tables?
It's better because if you have one application that has authority over that area, you only have to answer this question once. The answer will depend on the data in question, of course; maybe it is fine to serve stale data for a while, maybe you need to write to one DB, read from another, and combine in-process, etc., until the change fully propagates. But the impact needs to be localized to whatever extent is possib…
Over time they all learned to reach into eachother’s databases. The truth is we had ONE database arbitrarily divided into three schemas, each with different traditions.
As load increased it became a nightmare and a literal single point of failure. If one app misbehaved or took a load spike all the rest would slow/fall down. Even though huge chunks of the applications had nothing to do with each other they couldn’t be scaled independently.
We were working very hard, slowly, to detangle it without blowing everything up.
No application should ever have direct access to another application’s database. It’s going to go wrong. The temptation is too great. And by the time you realize it the technical debt it has caused may be MASSIVE.
Re: The Death of Microservice Madness in 2018
#45Biggest issue with microservices: "Microservices can be monoliths in disguise" -- I'd omit the can and say 99% of the time are . It's not a microservice if you have API dependencies. It's (probably) not a microservice if you access a global data store. A microservice should generally not have side effects. Microservices are supposed to be great not just because of the ease of deployment, but it's also supposed to mak…
That's plainly wrong. I get the gist of what you are saying and I more or less agree with it but you expressed it poorly.
Having API dependencies is not an issue. As long as the microservices don't touch each others data and only communicate with each other through their API boundaries microservices can and should build on top of each other.
In fact that's one of the core promises of the open source microservices architecture we are building (https://github.com/1backend/1backend).
I think your bad experiences are due to microservice apps which are unnecessarily fragmented into a lot of services. Sometimes, even when you respect service boundaries that can be a problem - when you have to release a bunch of services to ship a feature that's a sign that you have a distributed monolith on your hands.
I like to think of services, even my services, as third party ones I can't touch. When I view them this way the urge to tailor them to the current feature I'm hacking on lessens and I identify the correct microservice the given modification belongs to easier.
Re: The Death of Microservice Madness in 2018
#46Biggest issue with microservices: "Microservices can be monoliths in disguise" -- I'd omit the can and say 99% of the time are . It's not a microservice if you have API dependencies. It's (probably) not a microservice if you access a global data store. A microservice should generally not have side effects. Microservices are supposed to be great not just because of the ease of deployment, but it's also supposed to mak…
a bunch of API endpoints written by different teams is a "microservice architecture"
Or chaos, or madness, or Bedlam.Most people have enough trouble getting three methods in the same file to use the same argument semantics. Every service is an adventure unto itself.
We have a couple services that use something in the vein of GraphQL but some of the fields are calculated from other fields. If you have the derived field but not the source field you get garbage output and they don’t see the problem with this
Re: The Death of Microservice Madness in 2018
#47Biggest issue with microservices: "Microservices can be monoliths in disguise" -- I'd omit the can and say 99% of the time are . It's not a microservice if you have API dependencies. It's (probably) not a microservice if you access a global data store. A microservice should generally not have side effects. Microservices are supposed to be great not just because of the ease of deployment, but it's also supposed to mak…
> A microservice should generally not have side effects. That's plainly wrong. I get the gist of what you are saying and I more or less agree with it but you expressed it poorly. Having API dependencies is not an issue. As long as the microservices don't touch each others data and only communicate with each other through their API boundaries microservices can and should build on top of each other. In fact that's one…
I'm not sure what you think side effects are, but I'm using the standard computer science definition you can look up on Wikipedia. If you have a microservice that modifies, e.g. some hidden state, it's a disaster waiting to happen. Having multiple microservices that have database side-effects will almost always end up with a race condition somewhere. Have fun debugging that.
Re: The Death of Microservice Madness in 2018
#48Earlier quoted context omitted.
The problem is the codebases are still not "completely independent". If the backend dev needs to change a field in the database, or add a new one, then can might affect the front end dev. Also - simple things like validation rules - i.e. what can the front end dev allow users to store into the DB vs what can the back end dev use/need.
What would be an example of a situation where the backend dev needs to change a field in the database?
Re: The Death of Microservice Madness in 2018
#49Earlier quoted context omitted.
I've been down that road. Eventually it turns into a nightmare because evolving your DB means making simultaneous changes to multiple applications. "We think we can drop this column, someone figure out which of our eight apps using this DB might be using it still" I much prefer putting a single service layer in front of the DB that speaks thrift or protobuf and letting all clients interface with that instead. Evolvin…
How does an additional layer make changes to the DB easier? If a column is dropped what will your layer do when a request for data from that column comes in? What if the request joins it with other columns from other tables?
Let's take an address for an example. Your contact database might have "id", "name", "addr_line_1", "addr_line_2, "addr_line_3", "city", "state", "zip". Your API v1 lets you query any combination of these fields and return a set of fields.
/api/v1/contacts?zip=10018&fields=name
[{"name": "Jessica Jones"}, {"name": "Matt Murdock"} ]
/api/v1/contacts?name=Matt%20Murdock&fields=zip returns something like:
[{"zip": 10018}]
Later, you change your database to split name into fname and lname. You update your code and API to reflect this. In calls to /v1/ of the API, the field "name" is concatenated by your code to combine the database columns of fname and lname. In /v1.1/ of your code, you may or may not keep the "name" field as a convenience , but you do provide the ability to search on fname and/or lname fields.
Basically, for any change to the underlying database, your API maintains the same interface to the data, however you might also publish a newer version of the API. In fact, your code that handles the API might be entirely different. For instance, /api/v1 might be handled by a python flask framework, while /api/v2 is handled by node. You can run multiple versions of the API at one time, supporting multiple iterations of clients. Obviously to reduce the amount of maintenance efforts, you will deprecate some versions and provide deadlines for them to be disabled. You can run all sorts of reports on the usage of each API endpoint and see what calls and response formats are still in use. This can open up the ability to do an outreach to those developing a client and make them aware of necessary changes.