Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modules, not microservices

blogs.newardassociates.com

Post reply on HN
Print view View original

661–670 of 671 posts

Re: Modules, not microservices

#661

Post by rlt »

int_19h wrote: post #653
rlt wrote: post #590

Earlier quoted context omitted.

I’d love for a language / framework that allows for an application to be composed of “modules” that can either be run in a single process, or deployed as multiple independently scalable processes, with a mostly transparent RPC system requiring minimal boilerplate. My IDE should be able to easily traverse the call graph. My development environment should be simple to setup. I’ve worked on microservices that required a…

The industry has been trying to do "transparent remoting" for decades now. DCOM and CORBA were both basically that. It turns out that "transparent RPC" is basically a contradiction in terms. As soon as you start doing things across process boundaries, and even more so across network boundaries, it requires a very different approach for API design - something that's very cheap locally, like passing objects by referenc…

By “mostly transparent” I was mostly thinking type safety, ergonomics, no hand written boilerplate, etc.

Totally fine to support a limited set of primitives and collections passed by value, and require developers consider the failure modes.

In my experience HTTP+JSON or gRPC are usually lacking in at least one of those respects. If you constrain all the services to be the same language it becomes easier.

Re: Modules, not microservices

#662

Post by greenhatman »

SgtBastard wrote: post #647
greenhatman wrote: post #642

Earlier quoted context omitted.

I don't really think of route based "load balancing" as load balancing. That's routing, or a reverse proxy. Not load balancing. Load balancing is a very specific type of reverse proxy. The point is, if a client makes a request to a server, the response should always be the same, no matter where the load balancer sends the request to. Which means it should run the same code. Nginx doesn't even mention route based or e…

https://www.nginx.com/blog/nginx-plus-ingress-controller-kub... Describes exactly what I’m talking about.

That is still routing. Not load balancing. The load balancing is only between pods of the same service.

Just because Nginx is doing it, doesn't mean it's load balancing. It's an extra function tacked onto a load balancer.

Re: Modules, not microservices

#663

Post by mk89 »

rusebor wrote: post #643
mk89 wrote: post #630

Earlier quoted context omitted.

No, because that's not the service API. That's just a view over a table - an internal data structure used to represent some business domain model which should be properly exposed through some implementation-agnostic API. Service B should not care how service A implements it. And the fact you must keep backward compatibility ( see the OP answer above) at the implementation level shows how fragile this approach is - yo…

a view would allow you to change a field's type. something like this CREATE VIEW table_read_api AS SELECT TO_CHAR(now_int) as was_char FROM the_table; not sure that the view is an internal structure because it will be exposed via API as it is. SQL allows to swap the database at least if no specific SQL features are used.

> a view would allow you to change a field's type.

Which proves my point: if I change a field type ( in the table ), I will have to change the view type. I need to change 2 services because one of them changed an implementation detail.

Re: Modules, not microservices

#664

Post by randomdata »

Guid_NewGuid wrote: post #624
randomdata wrote: post #479

Earlier quoted context omitted.

> Whether it's your HTTP API interface changing Interfaces don't change under microservices. Communication by contract enshrines a contract. You must ensure that your API does not break legacy users no matter what changes you want to make going forward. You have committed to behaviour forevermore once you submit the contract to other teams. This may be another reason why IPC is often preferred over straight function…

Apologies for the delay in a reply. I think this has been a constructive discussion certainly for how I think about things. > Interfaces don't change under microservices. Communication by contract enshrines a contract. You must ensure that your API does not break legacy users no matter what changes you want to make going forward. You have committed to behaviour forevermore once you submit the contract to other teams.…

> Apologies for the delay in a reply.

Not at all. I'm in no rush.

> Secondly, what I think actually happens is that most companies using microservices aren't this disciplined and just end up bodging or implicitly coupling services.

If an organization is sufficiently large you may have no way to get another team on the line to even try. At that scale the other teams may as well work for other companies and that is what microservices models. If your organization is small, I tend to agree that you won't succeed. You can't beat Conway's Law.

> I find little value in providing that specification through the TDD approach or associated ceremony.

But, ultimately, what's the difference between writing your spec in an executable way or writing it in a Word document beyond the superficial differences in the languages? What is communicated to other developers ends up being the same.

> I prefer as few tests as possible at as high a level as possible with as few mocks as possible. I'll take a single test that takes 15 seconds to run over 1000 tests running in milliseconds.

Seemingly not all that common, interestingly, but Go comes to mind as a language that provides constructs to define separation between specification (TDD) and developer tests. Conceivably you could exclude your TDD specs from execution, only running the tests that help in your development process.

The value of TDD isn't in the execution, but in what is communicated to other developers. That the specs are executable is merely a nice side benefit to help with confirming that the implementation conforms to the spec. You still get 90% of the benefit of TDD even if you never run the executable, albeit granted at that point it is just a fancy Word document.

> I think there's a lot of healthy debate to be had about how much testing is needed.

Perhaps, but TDD isn't about testing. TDD is about providing documentation. I think there is less room for debate there. I'm not sure anyone who has ever inherited a codebase has wished it gave less insight into what the 'business needs' of the program are. Again, a Word document can provide the same documentation, but if you're going to write that Word document anyway why not go the extra mile and gain the additional benefits that come with execution?

Re: Modules, not microservices

#665

Post by mk89 »

jmchuster wrote: post #656
mk89 wrote: post #593

Earlier quoted context omitted.

> So then back to your original question, the way that this contract can break is via schema changes. So for us, since we use postgres, we created database views that we expose for reading. And postgres view updates are constrained that they must always be backwards compatible from a schema perspective. So then now our migration path is: > - service A has some table of data that you like to share > > - write a migrat…

if service B needs some new data from the view that isn't being provided, then you first run the migration on service A to update that view and add a column. Then you are able to update service B to utilize that column. If you don't need the new column, then you don't need to do anything on service B, because you know that existing columns on the view won't get removed and their type won't change. You only need to ma…

This only works if you apply backward compatible changes all the time. Sometimes you do want to make incompatible changes in your implementation. Database tables are an implementation detail, not an API which you're trying to expose as a view, etc.

But hey, every team and company has to find their strategy to do things. If this works for you, that's great!

It's just not a microservice by definition.

Re: Modules, not microservices

#666

Post by rusebor »

mk89 wrote: post #663
rusebor wrote: post #643

Earlier quoted context omitted.

a view would allow you to change a field's type. something like this CREATE VIEW table_read_api AS SELECT TO_CHAR(now_int) as was_char FROM the_table; not sure that the view is an internal structure because it will be exposed via API as it is. SQL allows to swap the database at least if no specific SQL features are used.

> a view would allow you to change a field's type. Which proves my point: if I change a field type ( in the table ), I will have to change the view type. I need to change 2 services because one of them changed an implementation detail.

before the change

CREATE TABLE the_table (was_char CHAR);

CREATE VIEW table_read_api AS SELECT was_char FROM the_table;

a client consumes data via:

select was_char from table_read_api;

after the change

alter table the_table modify was_char int;

alter table the_table rename column was_char to now_int;

CREATE VIEW table_read_api AS SELECT TO_CHAR(now_int) as was_char FROM the_table;

the client uses the same query

select was_char from table_read_api;

the type of the column has not changed

it is still a char

only "internal implementation" is changed

Re: Modules, not microservices

#667

Post by jmchuster »

mk89 wrote: post #665
jmchuster wrote: post #656

Earlier quoted context omitted.

if service B needs some new data from the view that isn't being provided, then you first run the migration on service A to update that view and add a column. Then you are able to update service B to utilize that column. If you don't need the new column, then you don't need to do anything on service B, because you know that existing columns on the view won't get removed and their type won't change. You only need to ma…

This only works if you apply backward compatible changes all the time. Sometimes you do want to make incompatible changes in your implementation. Database tables are an implementation detail, not an API which you're trying to expose as a view, etc. But hey, every team and company has to find their strategy to do things. If this works for you, that's great! It's just not a microservice by definition.

I would never claim that our setup uses microservices. Probably just more plainly named "services".

And yes, that is correct, we agree that once we expose a view, we won't remove columns or change types of columns. Theoretically we could effectively deprecate a column by having it just return an empty value. Our use cases are such that changes to such views happen at an acceptable rate, and backwards incompatible changes also happen at an acceptable rate.

Our views are also often joins across multiple tables, or computed values, so even if it's often quite close to the underlying tables, they are intentionally to be used as an abstraction on top of them. The views are designed first from the perspective of, what form of data do others services need?

Re: Modules, not microservices

#668

Post by nsmith7979 »

rlt wrote: post #590
barrkel wrote: post #72

Microservices, while often sold as solving a technical problem, usually actually solve for a human problem in scaling up an organization. There's two technical problems that microservices purport to solve: modularization (separation of concerns, hiding implementation, document interface and all that good stuff) and scalability (being able to increase the amount of compute, memory and IO to the specific modules that n…

I’d love for a language / framework that allows for an application to be composed of “modules” that can either be run in a single process, or deployed as multiple independently scalable processes, with a mostly transparent RPC system requiring minimal boilerplate. My IDE should be able to easily traverse the call graph. My development environment should be simple to setup. I’ve worked on microservices that required a…

I made a framework along those lines for work which we used for many years to mediate ~1000 namespaced commands across dozens of repos. Whether it ran distributed or bundled (I called it "mega-service" mode) was completely controlled by config. We mostly used ZMQ for inter-service messaging, but I had implemented http, redis pub/sub, grpc, etc at various times to prove the concept.

> https://github.com/NathanRSmith/lib-courier-js

Interestingly, we're pursuing a monorepo & multi-monolith setup for the next version of our platform. So lib-courier is no longer necessary to stitch it all together. It was fun while it lasted though. Once you understood the routing algo & code patterns, lots of stuff "just worked".

Re: Modules, not microservices

#669

Post by SgtBastard »

greenhatman wrote: post #662
SgtBastard wrote: post #647

Earlier quoted context omitted.

https://www.nginx.com/blog/nginx-plus-ingress-controller-kub... Describes exactly what I’m talking about.

That is still routing. Not load balancing. The load balancing is only between pods of the same service. Just because Nginx is doing it, doesn't mean it's load balancing. It's an extra function tacked onto a load balancer.

Friend, you don’t know what you’re talking about and if linking NGINX documentation literally describing load balancing algorithms applied across Kubernetes pods hosting endpoints doesn’t clear things up for you, I don’t think anything will.

Re: Modules, not microservices

#670

Post by ElectricalUnion »

catlifeonmars wrote: post #657
ElectricalUnion wrote: post #654

Earlier quoted context omitted.

> it’s literally a single point of failure. Those machines are made very redundant. All components in a Parallel Sysplex configuration can operate in redundant mode. Redundant power. Redundant central processor complex. Redundant multichip module (what would be a chipset on microcomputer platforms). Redundant RAM Memory. Redundant Storage. Redundant internal interconnects. Redundant network adapters. That's why corpo…

Redundant physical location. That means different power dependencies and different cities.

> Redundant physical location. That means different power dependencies and different cities.

Geographically Dispersed Parallel Sysplex.

Post reply on HN